Big Data Explained



hello this is toasting tomba architected IBM for data and analytics in the cloud and today i'm going to talk to you about server this technology and how it is applied to big data analytics when we look at big data in the past decades we can see that there has been it well there is a traditional form factor of big data systems that has been used for many decades already and as this the form factor of a data warehouse so this is a highly integrated system highly optimized for handling big data queries big data analytic in a very efficient manner nevertheless we had about it around the year 2000 Hadoop coming up and being adopted very rapidly and gaining a lot of popularity in this now widely adopted in the industry even though it also just big data analytics so why is that why Hadoop came up so this is because it brought in addition to this integrated system now more openness to the table more openness in terms of the type of data that it could handle data formats bring your own data formats the types of analytics analytic libraries and languages that can be supported and also the flexibility in terms of the hardware the deployment options that you can have you can bring a custom hardware even heterogeneous hardware so that's why I do basically gained a lot of traction and is now widely adopted today however we are seeing a trend that basically results in a yet another form factor of doing big data analytics and this trend is driven by actually one thing that is happening which is the error the raise of cloud and another thing that actually goes hand-in-hand a little bit with the raise of cloud is the consumption behavior of many people of end-users to be more oriented on a sharing economy so people are using more and more just write shares instead of just renting not to speak of buying a car just to get around or they're going with just Airbnb just to sleep a night somewhere so this consumer behavior is also applied now to IT and what actually this term server less is is actually exactly this so server less is in fact the sharing economy for IT and it is it is enabled by a cloud and it is in fact the most consequent usage model of cloud is serverless and many of you have heard the term server less and probably most of you will associate a thing called function as a service with serverless many of you think it's synonymous which is not exactly true but that's what's basically many people think of and functions the service is I have my coat and you need to run my business logic but I don't provision dedicated system that you get at hardware or not it's not even dedicated software I'm just sending it to a service say please run it for me running from me may be that many times so how to scale out it's all done ad hoc it's basically hiding the fact that there are servers that's why it's called serverless now as I said this is what many people think of when they heard room service but service is more than just function as a service especially when we now look at pick again at our domain here which is data big data and analytics the problem with big data analytics is that we are talking about state state has to be kept data my data has to be kept safely durable reliably I need to be able to access at any time when I want it and that's what these systems provide but now in the quality of new options we can actually abstract the storage of data itself as a cloud service on its own and that's also what's happening on the cloud and there is the basically cloud native storage of object storage an object's torch is basically serverless storage because you do not provision disk volumes do not configure disk volumes you just bring your data and the system figures out how to store it and how to basically also distribute it and make it highly available and so on it's highly abstracted you just have a REST API where you upload and download your data and you can come with kilobytes of data and going up to terabytes of data in the same organizational unit and the thing about now why is it turbulence is also that it's a pay-as-you-go consumption model you don't just use it as you go you also to pay-as-you-go which means you're just paying for the gigabytes that you're storing at this point right now and if you store less you will be paying us in a very elastic completely seamlessly elastic way now let me now talk about big data analytics it's not just about storage of data but also how can we analyze this data and process this data and that's exactly what we are now seeing as well driven by cloud we're seeing additional services that aren't available around objects torch such as sequel as a service or also it allows you to run SQL basically on the data object storage and just be billed for this one SQL depending on how big the SQL was in terms of how much data it had to skin and you do not pay for database that is provisioned and standing around it's just a single SQL and that's it and there are other things that basically play in like for instance messaging as a service so Kafka as a service where you're also just paying by the number of messages that are being processed and then eventually store to the object storage so there are a series of these services basically coming up and in combination they are providing this new form factor of a big data and analytics system that is augmenting and actually complementing the existing form factors because even though they are more established in order they are still a point for using them because they have to have sweet spots in terms of their own performance characteristics and response time guarantees but on the other side there may be maybe cost effectiveness benefits here so depending of your business model and requirements you may use this or this or the combination of those things so I hope this helps to put in perspective how serverless plays into big data analytics and how basically generates a whole new form factor of big data and analytic systems thank you very much

Be First to Comment

Leave a Reply

Your email address will not be published. Required fields are marked *