You’ve Got Key Values! A Redis Jump Start

About this course

and good morning from Redmond Washington thanks for joining us for another Microsoft Virtual Academy this time me and Rami are back for a redis jumpstart

what we're calling it you've got key values yeah it's the sequel so today we're here to talk about a no sequel key value store called Redis and a way to scale your applications and

make them faster more performant for really awesome applications and the awesome counter should be up soon so we'll see how we are at the end of this show but to continue Rami

I guess we can start with like what what is no sequel like what does that mean no sequel is this whole idea of not only sequel so traditionally we've had databases that were focused entirely on

having a type of tabular data store with a structured query language that was used to get data out of the database with the no sequel movement what we've started to see is different types of

databases take bigger part in the development cycle and different databases that support different ways of querying and not only sequel but sometimes very specialized languages so

give you an example there's Redis there's a Mongo that you've seen an MVA on already ready this is a key value store and it has its own query language that uses commands we will cover all

today and you've also seen a few graph languages as well and those are they have their own type of query language is very very very different from your typical sequel so that's pretty much no

sequel in a nutshell I don't know if you have anything else to add so I guess it's the idea that not sequels not just all and all type of datastore it's not like it's bad but it's just not the only

type and I guess that's where the name dusty go comes from right yeah cool so why don't we go introduce ourselves again for the folks do you haven't seen us before and we can go ahead and get

this ball rolling so Rob let's go introduce yourself first so my name is Ram UCR this is my third and the and I'm very happy to be here and I'm very happy that you guys are joining us

here today live from Redmond so I'm based out of Montreal I'm a Technical Evangelist so I focus mostly on helping developers code with the web html5 JavaScript nodejs ie11

Azure MongoDB and pretty much anything that falls within that realm of open source and web technologies so I'm based in Montreal I help startups there so if you're from Montreal please type it in

the chat room I'd love to – basically like find out who's actually listening from my home town I'm super active on Twitter so you can follow me on the a triamese there and if you're not in the

chat room but you feel like you got some feedback and you want to give it to me on Twitter go right ahead I will happily take that feedback you can also read some posts

that I make on msdn comm slash key need to have the links on the slides and I also blog on my own personal blog as well cool and so I might as well introduce myself – my name is Steven

Edouard this is I guess Envy and number four for me I'm a former software engineer software test engineer at run time I brought up the project and toolchain pluses dotnet four and four or

five one and four last year so I've been what you call developer evangelist and I've been focusing on startups in Silicon Valley area helping and enable them using Azure and other technologies

to really kind of jumpstart their businesses as well as you can catch me on my personal website at Steven Edward comm where you can find my thoughts and technical technical blog post so that's

a little bit about myself and so let's stop talking about ourselves and actually talk about what we're here for yeah so before we get started we might as well do a little bit housekeeping and

figure out what what we're gonna be doing today so you can kind of know when you want tune in or when you want to actually hopefully not too now alright so the first thing I'll do is talk about

intro no sequel then we'll go ahead and get started for ed us and I'll just be bootstrapping yourself getting going afterwards rami old continues to run section in the first three and we're

gonna do advanced data structures live Redis afterwards I'll take over with getting started as a rightist cast which is a fully managed as your service for Redis afterwards

we'll talk about reticent a variety of languages and then finally we'll finish off with Redis pop sub and actually scaling an actual application with Redis pub/sub capabilities so to set the

expectations what we would expect so just texted set I set up who the audience for this MVA would be a lot of startups like using this technology so if you're interested in building a

startup very quickly a lot of these technologies are you'll find a lot can you support for that open-source developers of course pretty much everything we're doing today is open

sourced and enterprise developers are looking to scale their applications or make them more performant suggestive prerequisites would be knowing a little bit basic knowledge on some programming

languages and some working knowledge of databases so there's not really too much for not going to too much for web programming today and so that's really all you need to get started with Redis

and before we get started go and let you know that Microsoft Virtual Academy community has what how many millions Barry says 2.6 million registered users which is really awesome and actually if

you actually enter the code as AZ are right it's cash you can earn points for this event for Microsoft Virtual Academy yeah get up on those leader birds on an MVA we've got plenty of videos on there

as well so if Redis is interesting to you perhaps you might be interested in also watching the nodejs MVA and the Mongo MVA that means to even actually did together yep okay so let's go get

start with the introduction to no sequel so essentially in this module what we'll cover is an introduction to no sequel what is no sequel what are the different types of no sequel databases out there

and then we're gonna turn that discussion into a tabular data sources and data stores taking a look at how data used to be represented in your database in the sequel era and then how

that same data model could be translated to the key value store error probably shouldn't say key value store era but he value store databases so that's per much the main outline for this module

and I guess we'll get started so what is no sequel at we might have answered that question a little bit earlier today jumped the gun a bit but not a problem let's talk a little bit more about it so

no sequel in terms of how our data model is represented is it the same consistent data model on all the different types of no sequel databases or does it change we've mentioned technologies like

MongoDB Redis are they like similar to tabular databases so that's a really good question so the consistency aspect in those sequel databases is is one for debate you'll have folks that really

advocate for enforcing schemas on the application level that means that the application so uses actual server is responsible for enforcing the schema as opposed to traditional sequel databases

which are restore the database server is actually responsible for enforcing the schema that's been defined so for example if you use MongoDB you don't really have that server-side enforcement

whereas something like my sequel Postgres you would expect to get more a more enforcement on that side which is really useful in some cases and this also can be stifling to for flexibility

in other cases in our case so we're using Redis today which is a key value store to where essentially everything is a type of key and it points to a value and in case of Redis it has a lot of

interesting types of values and interesting types of data stores or data structures that we can associate with those keys which gives us the really flexible power of Redis and Rytas

also kind of falls into the case where the the schemas really not enforced it's more enforced on application level and in a variety of these types of databases you have frameworks that actually are

make it easy for developers to enforce that that framework given that you're using that same thing that same thing that enforces a schema across all the applications that use that database

which could be challenging or could not be challenging depending your application yeah so I guess the next step is let's take a look at a very specific example of a data model so

we're going to talk first in how that data model looks in a tabular data store say we have a bank okay we're a bank were building our own bank and we want need to keep track of the the our

customers and the accounts that they hold so perhaps we might want to have a set of data representing the persons that are customers of this bank and accounts that they hold all right so if

you look at a sort of more traditional table a table or data store you might have two tables the first one being the account holders so these are all the customers of the bank and in this

account holder table you might have different rows and each row has an ID sort of a number to assign to this cuts this customer not that most banks thinks that they're there folks are numbers but

for the sake of the database we need to have an ID it's just the way that the technology is built and then perhaps we might have a first name and a last name so that typically will represent most of

the most important part of the account like the data of a customer the first thing my last name perhaps you might want to add an email phone number address but for the sake of our example

we're just stick to first name last name and then the other thing you might want to have is the accounts table and in this case our rows will be composed of an ID value once again and then perhaps

the account type let's say it could be an investment account it could be a savings account it could be a checking account the current balance and maybe the currency that is being used in this

account now what's interesting is well how do we know what what account is assigned to what customer right so that's where a foreign key comes into play so this is a link and a

relationship between the account and the account holder so what we have in this case on our table is the holder column which contains the ID of the account holder and that's how

we create a lationship between the accounts of the account holders so you'll notice that Stephen Edouard our account holder with the ID zero actually has three different types of accounts

with three different balances in this Bank so if we wanted to ask the question what is the name of account holder ig1 well in a type of their data store what we would do is we would look up the row

with the ID of one and then they would get all the information anything you'd like to add about this Stephen no not really it's pretty straightforward yeah awesome and then is it just feel free to

jump in anytime so I guess I didn't want to follow up and ask the question who is the account holder for account ID three all right in this case what we're looking at is the account ID of number

the account number three so you can see in our table that we've got that that fourth row so the one with ID equal three and then we want to look at who holds that account so we're going to

look at the foreign key assigned to that account in this case it's account number one we're gonna take that IG and then eventually find a way to find out which account holder actually has that ID of

number one now in tabulator store this is called the join so what you're doing is you're taking both tables you're joining them together at the point where the foreign key and the account holder

that's that's tied to it the relationship is put together yep so and this is basically so every sequel database has to do a join to accomplish this but one thing to note too is that

even in this case it wouldn't be terrible because the joints are on keys but if you're joining on things that aren't indexed as a key you you're pretty much gonna it can be quite

expensive especially as the data grows very large and that's kind of why they kill these heels of tabular data stores specifically sequel data stores because there's also

column-oriented data stores I don't do this so speaking of joins that's pretty much what has led to some of these new different types of databases that come

out at different ways to approach representing data and in this case we're using tables and rows we're using this operation called the join to match the data that we're looking for and a simple

answer to a question like which accounts are held by Steven and wired if we did that the join will have to take a look like will basically look like this we'll have to look first at all of the

accounts that have the foreign key for persons set to 0 because that's the account holder ID of Steven Edward so we'd have to do a query find out what that account ID is match it with all the

ones that have the same foreign key and then if you take a look at how you do that in an actual tabloid database that's also a join yep exactly and I think the thing to note too is that the

reason why that flexibility is lost is if you had deleted Steven Edward you couldn't do it until you delete the counts that steam that word has otherwise you'd end up with an

inconsistent database yes and then the database wolf will freak out at you and say no you can't do that because that's inconsistent so you have to delete foreign keys actually have enforcement

that it's you know that it's actually pointing to something all right so let's talk a little bit about how representing a very similar data model to this one would take place in a key value store so

a key value store essentially is a collection of keys and values so the data that's stored in there and Redis is an example of a key value store it's organized around the concept of an

associative array so think of a map think of a dictionary think of a hash if you've used Java or C sharp or Python or any there's a Java Script but any programming language and you have to

create an associative type of array that's exactly what a key value store is yep and essentially a key value store will contain collections of pairs key value pairs but the key

in this collection has to be unique so you could have key one value one key – value two key three value three and so forth it's fairly simple and then from this data model we can actually build a

lot of stuff on top of it you can build data structure on top of these key value pairs you can store lists as the value you can store set you can create sets hashes and a whole bunch of other stuff

and the key point here is that instead of storing data only into tables with columns and rows with key value stores you can build your own data structures on top of that very basic association

between a key and a value you can almost treat it as if it's memory right because what's memory is you've got an address a key for this memory bytes and then the value this in the byte so with a key

value store you can build very advanced data checkers on top of it Redis already has a whole bunch that are built in and we'll take a look at those in module 3 you've got to change your

thinking though so we in a tabular data store we had rows and columns now we need to find a way to translate that type of data into a key value and that's what we're gonna cover in this module

it's a radical idea so if you take a look at key value stores in you and you realize okay well key value right when do you typically need to know a value for a very specific key just to give you

an example of when this is extremely useful and the first and the best example it comes to mind is a cache so if you think about it if you have a few caching your website right and basically

a cache means that you're storing the resulting operation of some type of calculation it could be looking up the the actual like page in your web server it could be just storing the

intermediate calculation of a of a mathematical equation and you store that cache with the associated key in because you want to get the value right away and fairly frequently and you don't want to

keep redoing them basic calculation so that's essentially what a cast is useful for and in the case of key value store there well suited to operators cash because

that's essentially what they are you have a key and a value pair and that's it so if you know the key you can get the value right away so key value stores typically will not use sequel because

there's no reason to create a query language there's no reason to have this complicated set of instructions to tell the database how to get the data all you really want to do is get the value for

that key so this is like really kind of a kind of like if you could take it back to school it's like maybe a big oh right and it's like oh if you have Big O of a million things you have to actually go

through maybe and where n could be a million but if you have a key value store even if you have a million things it's still order 1 all right like just about like it doesn't really grow with

the scale of exactly happen there so if you use perhaps a hash or a dictionary in any programming language you'll know very well that to get the key you just say here's the key give me the value and

it's almost instantaneous there's there's no iterating through a list there's no iterating through a long table and that's really why key value stores are very high performance in

essence because at the very basic core it's just a key and a value that's it so you don't need to use sequel when you're building on top of a key value store and the other thing to note is that you have

a flexible schema you can put anything you want in the value you can just put a string that represents an object that's been serialized already or it could just be a number or a name of a person at the

end of the day your application layer is what decides how to interpret the data in the values that you're getting back so it's completely flexible you can do whatever you want to it that's cool so

if we look at our original example with the table tabular data store we have this idea that hey all of our customers could be represented in one table with the ID first name and last name as your

columns well that looks fairly straightforward so how do we represent that as keys well perhaps we want to say okay well the key is person this was my ID first name

Stevan person ID last name and edward that's the value so we've taken that first row and we've split the association between the ID and the value so ID zero and then first name Stephen

and assigned that its own key value pair so you literally just took all the columns and kind of we kind of put them together and that's what you actually use for the lucky so I basically have

split this row into multiple key value pairs yeah like ID and then first name is an ID and last name and then the model which is yeah person and of course you can choose your own way of assigning

the key so in my case I used the nomenclature of all the keys that relate to an account holder will start with person followed by a colon with the ID followed by the actual type of data

that's being stored in this case first name last name or whatever if we take a look at the more complicated table we'll notice it's not complicated in the sense that it's it's it's got a lot of

different components but it's just got multiple columns right here it gets a little bit more interesting but the very basic principle of assigning a key and a value is there so if we take a look at

the account ID 0 then we can assign a type to that account that's that would be the first row in the key value store section of the slide you'll take a look account 0 type investment account 0

balance the 80,000 same as in the top of the datastore and it just repeats now rami if I want to do a holder if I'd like to point to a account holder would I just add another key that points to

the key a great question and that's exactly it that's exactly the answer so if you take a look at the next slide and here's how we could describe the relationships like foreign keys with key

value stores so it's a little bit different in the sense that sequel and tabular their stories are built to these types of relationships all over the place with key value stores because

they're operating out with this concept of just key and value only it's the application layer that the science how to interpret the value okay so the application layer could decide that this

value is actually describing a relationship to another key that's in the key value store so so they can look at precisely how do you put how do you link an account holder with the account

one way to do that is actually to say hey the key is account ID holder and the value is person zero that's my key okay so if I take the value and then I add either a first name and last name or

anything I want I can literally take that string and then give it as give it as a key to the key value stores to get that value so I guess the thing to note here is that it's work it's necessary to

do multiple interactions yeah get to that get through that relationship so you can't just do it in one yeah you can't quite build a query that just gets you the data right away but

it's still it but I think the important point here is to note to note is that even though it's a key value pair you can still build on top of that relationship and that's it's going to be

a common theme throughout the envier we're gonna see how Redis builds on top of this concept of key value pairs to have more advanced data structures among them is a hash in Redis and what a hash

does it's it's literally just describing an object with a single key so you can say hey this is my key here are my key value pairs and it's ready it's already builds a data

structure on top of the key value pair so that you can do that very efficiently with very simple commands now of course the questions become very interesting when you got to ask okay well which

accounts are held by Stephen Edouard now our relationship was originally described as the count specifying who is the holder of this account now we want to do the opposite we want to sort of go

and create a reverse index here now obviously there's two different ways to actually get the value you could scan all the key values and if you if you scan

all the key values you're basically going through all of the keys trying to find the right answer yeah and that's an Owen operation because it's just you're basically going through the entire list

right alternatively option two which is my preferred option is you actually create a reverse index now we've mentioned earlier that you can build lists on top of key value pairs and you

can put any value you want in the values part so in this case what we can do is we can take a look at the person's keys and say hey person 0 accounts here's a list of all the accounts that this

person holds and that helps us answer the question of which accounts are held by Stephen Edouard without having to scan through the entire list of key value pairs cool

so where do key value stores shine and that's a that's a great question right so key value store is at the other fundamental level it's just a collection of key value so it's basically you can

basically treat it as if it's memory right that at its most basic level that's what it is it's it's like the RAM that's in your computer it's every single cell that's

in your RAM has got an address that's your key and whatever byte is stored in that in that cell that's that's your value like right away the the classic example of when a key aware key value

store shines is a cache and a cache fundamentally as we said let's you just get the value for a specific key but what's interesting about Redis is that you can actually take that same memory

and account I'm sorry that that same memory like the whole idea of having a key and a value yep and build stuff on top of it so you can build things like lists what's the list it's a bunch of

keys and values and conceptually the way you could think about this is here is my list all right Mikey would be list and then the end

for the first element my list would just be following that so I would say list 1 : 0 the value that's in that key is the first index in my list ah that's clever yeah so you you can build

very interesting things on top of lists right and the other thing that you could do is top of key value pairs like that it's just basic memory it's just this whole idea of having a key and a value

you can also build things like sets which is basically a list but where all the elements are unique right so as opposed to having repeating elements you would you definitely only have only one

unique element where you could do things like set theory yes okay oh you can do things like set theory so if you remember from your CS courses if you've taken a CS course set theory is a big

deal because it's it's it's also the basis of sequel as well yeah so Redis is a high-performance key value and data structure store and we say data structure store because it doesn't just

support lists and sets it also supports things like sorted sets hashes hyper log logs it there's a whole bunch of stuff and in essence Redis is a very popular cache because all key value stores can

be used as cache so lots of folks have actually taken by this and they've used it as a cache to replace memcache perhaps you've heard of if you're a web developer or some of the other

proprietary cache solutions Redis has become the more popular option right now and it's also an open source project so you can always edit the code you can improve it you can make it work on your

specific hardware you can tell it to take advantage of the caches that are in your CPU as well you can build for that and also cross-platform and it's cross-platform it's a it's a great piece

of open-source technologies and the other thing that it also does is it supports publish/subscribe messaging now here's something that's interesting what is that what is that and where did this

come from all of a sudden to a key value store so if you take a look at what is a pub/sub Mac like what what is pops up it's the idea of having a list that people can publish

to they can add elements to and a bunch of subscribers that are taking things off that list okay now normally you would call it list like you write because it goes in from

one end and it comes out from the other side right for first in all right first out right so if you take a look at Redis if you know that you have a key value store that's very high performant and

you've got this memory that's right there and it is very high like it's very fast like it's you got the key and the value and you can create lists on top of that you couldn't take that a step

further and use the list to build a queue right and then if you take it a little bit further you have a publish/subscribe mechanism right away because you've already got the queue all

you need to do is say hey Rhett is put stuff at the beginning and hey Redis give me whatever is at the end of this list so pop in and push interest and essentially that what that is what the

developers have writers have done they've built a pub/sub mechanism on top of Reddit so that anyone can actually use Redis as not just a key value store in a cache right but also as a messaging

platform nice so you could actually interactive messages yeah cool cool yeah and that's something actually that we're gonna cover in the last module we're gonna show you how to actually build and

scale an existing chat application with Redis sounds exciting yeah it does so you've got to stay tuned for the next module so in the next module what we'll do is we'll show you how to set up Redis

on your machine we'll teach you some of the common reddest commands that are out there so how do you actually set a key value pair how do you check to see if it exists how do you get it how do you app

into a list how to create a list how do you create a whole bunch of we're doing series yeah when we're doing sets as well we're doing sorted sets so you definitely gotta stay for the next

couple of modules and we'll teach you a little bit about the rightest data type so what data types are actually available and native within value the value part of the key value store

you'll learn a little bit more about ready strings and lists as well and of course any good cash will have u expiring key value pairs the time to live you need a time to live like you

don't want to like let's say that you're cashing a mathematical operation depends on time well that time is going to be obsolete fairly quickly let's say that you your calculation is only valid for

the last hour if that hour expires it's you got a you got that you have that data disappear so that you can the next time that you ask for it you can actually say okay well I can't find it

in my cash I got to recalculate it yep I'll go deeper into that later yeah so definitely stay tuned and if there are any questions please feel free to add them into the chat room we'd love to

take them at anytime cool so I think that wraps up module 1 introduction to no sequel and I forgot what else we called is that what is that what it's called I forgot hey guys

that's module 1 stay tuned and hold on with us we'll take a short time at a break and we'll be back for module 2 or we'll go ahead and start on getting started for bredis and how to get

everything bootstrapped and gone thanks for joining us we'll see in a sec