Up next in 10
Multi-Cloud Magic: Leveraging Multi-Cloud Clusters in Real-World Scenarios || Women Data Summit 2021
3K views
Nov 9, 2023
Public, Private, Hybrid...Multi-Cloud? As the cloud development options and architectures continue to evolve, we've finally found ourselves at the next new thing: Multi-Cloud. What is it? Do we really need it? How would you take advantage of a multi-cloud solution? In this talk, I'd like to describe a few real-world use cases where a multi-cloud solution is currently in production, specifically when it comes to data. A demo on how to setup a multi-cloud cluster will also be shown. The audience will leave with a more confident understanding of real-world, multi-cloud use cases and see how to create their own! Conference Website: https://globaltechconferences.com/event/women-data-summit-2021/ C# Corner - Community of Software and Data Developers: https://www.c-sharpcorner.com C# Live - Dev Streaming Destination: https://csharp.live #Data #WomenDataSummit #MultiCloudMagic #Clusters
View Video Transcript
0:00
All right. Hello, everybody. I had the great intro, so I don't need to introduce myself
0:08
But today I would love to talk to you about multi-cloud magic, or in other words
0:14
how to actually leverage multi-cloud data clusters in the real world. Because sometimes
0:20
when you see these kinds of solutions and where you see them in production today
0:24
it can kind of seem like magic, especially when compared to what we've seen before
0:29
So let's get into it. We'll first start with what exactly is multi-cloud because we've kind of heard
0:36
this word tossed around a lot and we want to get on the same page about what it means for the
0:41
context of my presentation at least. And then we'll get to the next question which usually comes up
0:47
in most people's minds when they hear about multi-cloud is do we really need a multi-cloud
0:52
option? And then we'll get into the bulk of this presentation. We'll go through some actual real
0:59
world use cases of how multi-cloud is being used in the real world. And then finally, we'll see how
1:05
easy it is to actually set up your own multi-cloud cluster. So what is multi-cloud? And if you're
1:13
thinking about it, and if you just look at the words, it seems pretty self-explanatory. Don't
1:18
doubt yourself, because it's probably the right definition in what you're thinking. Multi-cloud
1:23
at least for our purposes and for the most generalized definition is anything any single
1:29
architecture that uses two or more cloud providers and in this particular case we're going to focus
1:35
on currently the big three so that's AWS, Microsoft Azure and Google Cloud Platform or GCP
1:43
and that's really it that's all you really need to know. So if you are having any kind
1:48
of application that might be using both cloud providers or three cloud providers, you are using
1:54
a multi-cloud architecture. So now the next question comes up because, you know, as with most
2:00
new things, people like to poke around and say, well, do we really need this? Maybe this is just
2:05
a fad. And we want to take a look at if we actually need this thing, if this is just some
2:11
something that is another trend. And if we take a look back at the last few years
2:19
we'll see that there's been this common pattern with most of the cloud providers. So for example
2:25
in June of 2019, there was what Wired Magazine called the Catch-22 that broke the internet. And
2:32
that's, you know, that sounds super scary, but that's pretty much the scale of how this outage
2:36
affected the rest of the world. But what happened here was that Google initiated what should have
2:43
been a routine configuration change. It's like a maintenance event that's normally supposed to run
2:49
and it was only intended for a very specific geographic region. But, or when that happens
2:56
what happens is Google routinely reroutes the jobs that those servers are running where the
3:01
maintenance event is about to occur, and it reroutes it to some different machines
3:05
Well, what happened was, with a combination of things, right, but mainly two misconfigurations
3:12
and a software bug with the automation software, it led to a bunch of networking jobs being
3:18
descheduled in multiple locations. And so if you want to think about what that actually means, if you think of all the data
3:25
that Google, you know, runs through every single day, and you think of the normal amount
3:32
running through maybe like six tunnels or something. When this happened, all of those
3:37
tunnels that the data was able to go through maybe got cut down to like a third. So if it
3:42
had six or eight, six tunnels, then you had two left. And so this effectively resulted
3:50
in an internet wide gridlock. But it's not just GCP. There was also an AWS outage in October of 2019. In AWS's own
4:02
retrospective, they explained what caused this. And it was basically a failure that happened with
4:09
their data center control systems. So these systems are used to automate the cooling and the
4:17
balance in the data center. And all of this was running some third-party code
4:25
which allowed it to communicate with the different devices in the data center and the
4:30
temperature sensors, the fans, the chillers, all that stuff. So again, due to another bug that was
4:36
part of this third-party control system, that exchange pretty much resulted in like a lot of
4:42
interactions happening between this system and all of the devices. So it became unresponsive
4:47
because it was just attacking and sending all of these exchanges 24-7. And what that ended up doing
4:54
was it started overheating a bunch of racks of servers and it caused a bunch of redundant cooling
5:00
systems to fail. So that was the cause of that. But what we see here is that another large cloud
5:05
provider, AWS, also had their own outage. And on and on, right? Google Cloud had another one in
5:13
March of 2020. This time they had a router failure in Atlanta. And then Microsoft Azure also saw some
5:21
shortages happening with their data centers in Europe, especially when the pandemic hit. When we
5:27
first started to all go work from home, a lot of bandwidth was being used. We really started to see
5:34
the limits of what was happening when all of this additional unexpected traffic started to occur
5:41
And just as an example, Microsoft shared that when everybody started to go home during this pandemic
5:47
and started to work from home, there was a significant spike in Microsoft Teams usage
5:52
They basically had 44 million daily users at one point, and that generated over 900 million meeting and calling minutes on Teams in a single week
6:04
So you can imagine that anytime we get any kind of unexpected traffic, it's always like, you know, we're all kind of scrambling to make sure to accommodate that
6:12
But that was just a whole nother level. And the story keeps going, right
6:16
It's not tied to any single one. Azure goes down in the APAC region. And then the most recent one that was fairly large was AWS
6:25
in December or November of 2020 when they had the outage with the Kinesis data streams
6:31
And even if you Google now, you know, Google cloud provider outage, you'll find that there's
6:37
always something that was down, something that was, you know, out for maybe a day or two
6:42
or people can't access websites. And even if you look at your own history, there are times where
6:48
you know, we just can't access something that we need to. And so when you understand that
6:53
when you think about that, when you see this timeline, and you even look back at your own
6:58
experiences, the one thing to point out about that is that, you know, yes, we know that there are going
7:05
to be outages, even though we don't want them. But we do kind of expect them, right? It's not
7:10
something we think is impossible. But the bigger thing to see here is that at least right now
7:16
and what we've seen from history, there's no cloud that is spared from these outages
7:21
Every single one that we seen of the big three has suffered some sort of outage And so keep that in mind as we keep thinking about why multi might be an effective solution to this
7:35
Because when you keep that in mind, that goes very well into our next section
7:40
As we start to see this, we found that there was this think tank that occurred
7:45
That was basically this CIO think tank brought together 30 of the top IT leaders to talk about these actual problems
7:54
and specifically if something like a multi-cloud architecture would be able to help or even
8:00
make sense or fit into their industries and their specific architectures. So this included executives and CIOs from like Dell, Allstate, Visa, MetLife, all of
8:13
these different companies to kind of sit down and talk about it
8:17
And what resulted from this think tank is that they all pretty much agreed
8:22
It's not a matter of if, but a matter of when they were going to slowly start adopting and
8:28
incorporating a multi-cloud strategy into their fields. Now when this was published, there were a lot of really great quotes that summed up what
8:37
the feelings were about why multi-cloud is a great solution to kind of keep researching
8:43
and keep seeing if it would fit into their architectures. And I apologize if you hear the doorbell
8:50
One of them here is Gregory Scherr, and that is a VP of business platform technology of Fiserv
8:57
So what they do is it's a Fortune 500 company that's known for providing financial services
9:03
And what that means is that their kinds of clients are banks, their credit unions, their leasing and financing companies, retailers
9:12
So that means something like this. What Gregory Scherr said is the main driver is what our clients are asking for
9:20
We have banks who have an Azure preference. We have banks who have an AWS preference, Google Cloud, and on and on. We don't really get to choose. And that's incredibly key for companies like this. They are almost the ones in the middle where they have to accommodate their clients
9:38
If their clients have a requirement that says, oh, we are only able to host our data on Azure, you need to be able to accommodate that if that is your kind of client
9:49
Or others just, you know, have a preference. And if you want to expand your client list or be able to service a larger network of clients, then having this capability to be able to host their data where they need it or where they are required to have it is incredibly key and a really good indicator of why multi-cloud is something that is going to be more and more prevalent in the future
10:12
Another really, really great quote that I like is from Mohan Pucha, who is the Vice President
10:18
of Architecture and Digital Strategy at Aon, which is another kind of professional services firm that
10:24
sells financial risk mitigation products. He says, we have to be native AWS because of their advanced
10:31
capabilities in ytics, and we have to be in Azure because frankly developers love that ecosystem
10:38
And productive developers are probably the best thing. So that's why I like this quote, because whenever you hear about these kinds of solutions or you hear about these, you know, next new things that's going to solve everything, it usually does come from the top down
10:54
It's kind of like this is the decision that has been made and we're all just going to follow it and implement it
10:59
But what I like about this quote is that it explains that multi-cloud is not just this kind of overarching top-down solution that we have to implement
11:07
It's something that actually empowers us as devs because as we're starting to become more autonomous, as we start to be able to make more of these decisions, especially at the architectural level, multi-cloud actually opens up a bunch of solutions for us
11:24
And we'll see that in a moment. But it basically expands our toolbox as devs to be able to use the tools that we want or the tools that are the best for the job
11:34
So now that we have an understanding that outages are going to occur, no cloud is spared from
11:42
it, and multi-cloud is pretty much just making use of the cloud providers that are available
11:48
to us, let's see how that is actually being used in the real world
11:52
We'll start with a very, what I like to call the low-hanging fruit or the one, the scenario
11:59
that makes the most sense and is easiest for companies to see why this is beneficial for
12:04
them. So here we'll take Canada and in Canada, there's a specific direction for electronic data
12:12
residency. This is part of their strategic IT plan, one that they create every few years or so
12:18
But it's basically a list of best practices that they give any government entities or public
12:24
entities in Canada to follow to make sure that their IT infrastructure is sound, is secure
12:31
and all of that stuff. So one of those directions as part of that plan is this. It's the direction
12:37
for electronic data residency. It's a tongue twister. And that basically says any sensitive
12:44
data that is collected or used by the government of Canada has to be stored in an approved
12:51
computing facility that's within the geographic borders of Canada. So there are a few exceptions
12:57
You know, there might be some diplomatic places or consular missions that are outside of Canada, but those are very rare
13:04
And for the most part, most of the data or all of the data has to stay within Canada
13:11
So if we take a look at that, there are specific provinces that apply this in different ways
13:16
The British Columbia, Nova Scotia provinces, they take that to mean all of their data, any data, whereas Ontario focuses more on health care data
13:26
And that brings us to the very specific client that we're going to talk about today
13:31
So if you imagine, all of this data is stored in AWS Montreal, because that is actually the only region in Canada for AWS
13:43
So they kind of have no choice. If you're going to be using AWS, you're hosting your data in AWS Montreal, and that's it
13:51
Now, this is really, really scary. And for the most part, it's not. But we've seen some scenarios of why this is starting to become scary, specifically with this scenario
14:04
It was an emergency services application and they needed to, you know, an emergency services application is one kind of application that you almost never want an outage for
14:16
Right. So if you think of any other application that we've dealt with, you know, outages are always unacceptable
14:21
At the very, the lowest end, where the least impact, it may be an inconvenience to us
14:27
At the most extreme impact, like with this company and this type of application, that
14:32
could mean actual loss. And so when you take that together and you average that out, what it comes down to is
14:40
any kind of application downtime almost always translates to either financial loss, brand
14:46
or reputation loss, or something much worse that we don't really want to occur
14:49
and of course when we looked at the timeline of all the recent outages that occurred the recent AWS outage did not help it did not make this company feel confident that you know AWS Montreal was never ever going to go down So in this particular case and because there was no
15:07
proper failover strategy or no other region to kind of failover to, if AWS Montreal were to ever
15:13
go down, then so their application. So how do we fix this? Well, something like a multi-cloud cluster
15:21
is a great solution for this particular thing. Why? Well, to start, we could take advantage of GCP Montreal region
15:30
That's the only other one that's in Montreal. And in this case, if AWS Montreal ever were to go down, it could fail over to GCP
15:39
But to make sure there's additional fault tolerance and to really make sure this
15:43
never goes down, they're able to also take advantage of Azure's two regions that are
15:48
close by. So there's one in Quebec, which is Azure Canada East and one in Toronto
15:53
which is Azure Canada Central. So now this kind of architecture, having a, having your data hosted in a
16:00
multi-cloud cluster across three clouds is incredible because as far as we've
16:06
seen today, there's nobody else that can do this, but also this protects you
16:10
from most kinds of scenarios. So now let's say AWS Montreal were to go down
16:15
well, they can fail over to GCP Montreal and they can step up and fill in the gap
16:21
And if for some odd, rare reason that Montreal as a whole just goes out
16:27
that's when the Azure regions can step up and fill in the gap and ensure that there is no outages that occur
16:36
So that's kind of the most common scenario that we'll see. Making use of all three cloud providers regions is something that is so key to help us in fulfilling these kinds of data residency requirements
16:51
But it's also great for giving your application higher availability and fault tolerance that you might not otherwise have had if you were just kind of focusing or relying on a single cloud provider
17:03
Another really quick example of this is in Australia. So they had a very similar piece of legislation that says, or it was called the My Health Records Act of 2012. And it's the same thing. What this means is that any information of their residents that is related to health care, if they were processing it, storing it, had to remain within Australia
17:24
So if we take a look at the landscape of Australia, it's fairly similar, and you'd be surprised at how many other places are kind of like this
17:34
When you look at your options, you're pretty much stuck with this
17:39
You have GCP Sydney and you have AWS Sydney, and that's about it if you were to use those cloud providers
17:46
So in this case, Azure comes to the rescue again because they do have more regions available to offer any companies that have to abide by this, especially if they're healthcare companies or deal with any kind of healthcare data
17:59
So in this case, they can take advantage of Australia Southeast, which is based in Melbourne
18:04
and specifically in Canberra, they also have two regions, Australia Central and Australia Central
18:11
too, because they found that there are actual additional requirements where they needed to have
18:17
in-country disaster recovery. And so those two regions were actually built to help accommodate
18:22
those very specific scenarios. But again, it's very similar to the Canadian example, right
18:27
If you were stuck or were not stuck, but if you were on one of the other cloud providers
18:33
needed to have that, again, a failover strategy that was outside of just one city
18:39
you would have to take advantage of other cloud providers. And so, again, this is another great example of why a multi-cloud cluster
18:46
is something that is so key for this kind of solution. And just in case you're wondering, Azure also has one in Sydney
18:54
So kind of like Pokemon, if you wanted to catch them all. So those are the very, like I said, low hanging fruit, the easier to understand and very logical
19:10
solutions of why a multi-cloud cluster would be beneficial. But now we'll take a look at some
19:15
other scenarios where it has also been very beneficial for these specific companies
19:21
So this next one I get to call the best of all clouds, and this circles around this recommendation feature
19:29
So what we had was a customer that was running an application, basically like an internal type of help desk software
19:38
And all of this production traffic and load was currently working off of AWS
19:44
The data was hosted off of AWS. As they started to expand this application, they actually wanted to use or they wanted to build in some sort of machine learning into their help desk software
19:58
They wanted to be able to refer some knowledge base articles that might help their IT help desk technicians to better solve the problems quicker
20:08
So if they see that there's a pattern, they want to be able to recommend, hey, maybe this knowledge base article might help
20:14
So after talking to their devs and researching, they found, we actually want to use Google's AutoML, which is a natural language processing tool, and it uses machine learning to reveal the structure and meaning of text
20:30
So that's what they've settled on. That's the one thing they wanted to use to build this recommendation feature
20:36
Now, the big question here is, well, data has to get over there somehow, right
20:41
we need to feed this ytics application to be able to use AutoML. So how do we do that
20:50
Well, one way we have solved this problem is to write more code because, you know, it works
20:58
right? We write custom code, we build a unique data pipeline, we build something super unique
21:04
for this thing. But that's also kind of a problem. Anything that we build that is so bespoke, that's so
21:12
tailored to just this specific problem, it almost always makes it much harder to either refactor
21:18
later on or to remove and replace with something else. Plus it's also something, a separate thing
21:24
that needs to be maintained. And in this case, this client did not like that option because it was
21:30
another piece to maintain. It was another thing to keep track of. And even as devs, even if we
21:35
tried to automate this, would say something like Kafka, which is something that streams updates
21:39
from one source to another. It was, again, it was still something that was a separate piece that this
21:44
particular client did not want to host and did not want to maintain. So what's another option to do
21:51
this? Well, something called backup and restore. In this case, we would have snapshots taken off of
22:00
the production data from AWS, their live data, and then restore it to the database on GCP
22:07
where it would be used and yzed. Again, though, the client did not like, they tried this
22:13
and they did not like that it was costly to maintain this type of ETL process, this extract
22:18
transform load process, because it happened in batches. And because it happened in batches
22:23
the ysts that they had that were trying to use this new ytics application to feed the knowledge base articles We always constantly waiting for new data to be uploaded So in this particular case
22:38
a multi-cloud cluster was what worked for them. So why does this work for them? Well
22:44
the really big thing is that they were using the same underlying cloud database. So in this
22:49
particular case, they were using MongoDB Atlas to host their data, but the key here is that
22:54
it was able to support both their operational workloads, the ones that were running off of AWS
23:00
and their ytical workloads, which is the new thing that they wanted to add on GCP
23:05
And the way they're able to do this is this cluster actually hosted nodes on both clouds
23:12
So they added a read-only ytics node, which is made for things like long-running queries and
23:19
ytics and they would use that to feed their ytics application at the same time all of their
23:26
aws nodes their production traffic nodes were kept separate so all of that traffic was not
23:31
competing with their ytics node they were focused on just servicing the internal help desk
23:37
software application while the ytics node could be used solely for feeding the auto ml
23:43
ytics application and kept it separate. And that was what made it so great for their
23:50
particular application and how they built this recommendation feature in. Another really cool example of this, which is something we don't often hear about as developers
24:03
is sometimes using multiple clouds is something of a cost thing or a cloud arbitrage thing
24:10
And in this very extreme example, what happened here was we have an auto manufacturing company, we'll say, and they had most of their application also running on AWS at the time
24:22
But as they started to grow and needed to run more workloads, what they did was they spoke to both AWS and Azure at the same time
24:31
And what happened was what they wanted was the ability to, let's say they negotiated a better price with Azure, for example
24:41
They wanted to be able to say, I want to now move over to Azure because I have a better price there
24:47
And, you know, this seemed out of the ordinary. This seemed impossible
24:51
But again, with something like a multi-cloud cluster, that is possible because what would
24:57
happen here is you just need to change the highest priority provider of your data
25:03
So, and you know, this might seem like just a slide where I just put the name there, change
25:08
provider, but that's literally what you do in the Atlas UI. And we'll see that a little bit later
25:13
So if this were to occur and, you know, they said, Hey, we need to move over to Azure right
25:16
now, they could do that. They would change the provider to Azure. so they would start the rollover process where they would start migrating over all of their
25:24
hosted data from AWS hosted nodes to Azure and then be fully on Azure
25:30
And that's another really, I thought was very interesting because we don't kind of, we usually
25:34
don't see those kinds of things, but that is another thing that is thought about
25:40
It's another ability that would make multi-cloud clusters even more key and tempting for these
25:48
kinds of solutions. So, those are kind of the best of all clouds type of things, but what about developers
25:55
Remember I referenced Mohan Poocha and how he gave a really great quote about how having
26:01
the ability to choose whatever cloud we want is also a great thing because we're the ones
26:06
that build this thing, right? So how does multi-cloud affect us? Well, if you ask any dev, they're going to have some preferences and they're going to
26:16
have some opinions and they're going to say there are really, really awesome tools for very specific
26:21
things. Some people swear by AWS Lambda functions, or I actually like Azure functions, just like
26:28
Anna was mentioning in her talk right before me. If there's anything that's dealing with machine
26:34
learning or artificial intelligence, most people gravitate towards GCP's platform because they just
26:41
like working with it and it has either the most data or it is, it works the best in what they want
26:47
But that's not to say that those are the de facto standards. That's just to say, usually when we are coming up with solutions for building things, sometimes we either have to stay within the cloud that we are using or our company is using
27:02
And sometimes the tool we want to use is on a different cloud. But with something like a multi-cloud cluster where our data is hosted across all three
27:10
or on the cloud providers that we need, we're able to take advantage and actually use the
27:15
tools that we feel is best for the job. Another thing as developers is we're now responsible for even higher availability and even lower latency
27:26
So as we start to become more and more global, we are expected to make sure everybody has
27:32
the same experience on our application. We want to make sure that everyone gets quick loading pages, that things are working the same way as it would in a place that has higher network connectivity or more regions to work with
27:48
And again, as we've seen in the Canadian and Australian examples, sometimes that's going to require us to expand out and make use of two or all three of the cloud providers
27:59
Because if you take a look with these rough numbers, as of the time I researched, they're always growing and expanding and adding more regions
28:06
But when you add them all together, well, you pretty much have a really large share and an awesome variety of regions that you're able to choose from to help deliver and accommodate and implement and make sure we achieve that expectation of being able to give all of our customers the same experience when using our application
28:30
And then finally, we are striving to be cloud agnostic. What this just means is as we are starting to grow and starting to use more cloud providers or even just one and migrating things to the cloud, it is going to continue to evolve
28:46
And the best thing that we can strive for is to be able to be on the cloud that our customers are on or where they need to be and be able to be as agnostic as possible
28:57
so that all of the scenarios I showed you prior, whether that's because of cost or because of using the best tool
29:03
or not having vendor lock-in, we're able to move just as swiftly as we need to
29:10
So that's the developer's choice. The last thing and scenario I want to show you
29:15
that has made multi-cloud clusters a really popular option now that it is possible is something called future-proofing
29:23
and specifically with companies that deal with mergers or acquisitions. Another really common example is there's
29:32
a European company that acquires another company. Usually, most European companies that we've seen are fully involved in Azure
29:44
Their whole ecosystem is Azure, they're 100 percent in. Whenever they acquire a different company
29:51
and most of these companies are on AWS, they need to perform a cross-cloud migration
29:56
So again, there's this problem that needs solved where they need to migrate all of that over
30:03
So of course, how do we solve this? The first one is, of course we can write more code, right
30:08
But we know all of the drawbacks to that. We know anything that's super unique and bespoke and just purpose-built is not
30:16
usually the one that we want to go to. It's there. It's always there. It's always going to work, but we'd rather not do that
30:22
Now the second option, very specifically for this client, because they were using MongoDB Atlas
30:30
which is our database as a service, they use something called live migration
30:35
So what that does is very similar to the snapshots, but basically you have the data on AWS
30:43
but the problem is you have to also set up a destination cluster
30:47
So you'd have to set up another node in your destination Cloud provider
30:53
which in this case is Azure. The biggest issue here that we've seen is that even though it goes
30:59
over and migrates the data, not too bad. The biggest issue is that the connection string
31:06
needs to be properly bounced to make sure that your traffic was properly cut over and is now being
31:11
directed to the new Azure notes that were just migrated to. If you've ever done something like this before
31:19
that's the worst feeling ever where you know usually this has a lot of logistics around it
31:23
there's a lot of you know custom scripts that do often have to be run and that's the worst
31:29
thing to find is after all doing all of that and seemingly checking off all the things on
31:34
your list to find that your new production traffic is not actually going to the right
31:38
place it's still going to the wrong one so that was been the biggest annoyance really and
31:44
inconvenience when using this kind of option. As I've alluded, because this customer was already using
31:53
MongoDB Atlas, in this case again, multi-cloud cluster was something that worked really
32:00
really well for this cross-cloud migration. Because they needed to migrate over AWS over to Azure
32:10
they just needed to start this rollover process on that same cluster. Now how does this work
32:17
At a minimum every cluster is always three nodes. There's always two secondary nodes and there's one
32:23
primary node. And when this cross cloud migration occurs, when they start this
32:28
they obviously want to do this in a graceful manner. So what happens is that they would
32:32
first start migrating over one of the secondary nodes, then, or two of the secondary nodes
32:39
Then they would elect a new primary with the goal that the new primary is now going to be on the new destination cloud provider, in this case, Azure
32:49
And then finally, they would migrate over the remaining secondary. And then that's how it goes over and is migrated in a graceful manner, in a rolling manner, without downtime, over to Azure
33:02
But the best part about this is that the connection string does not need to change And that was the biggest problem with the last solution is that because of that because of having to manually assure that that connection string was cut over properly
33:16
you had that possibility that your projection traffic was not now going to Azure
33:21
But in this case, string doesn't change, which means it should not result in any kind of downtime or any kind of weird traffic issues being routed to the wrong place
33:32
And so that's kind of a smattering of all the real world use cases where multi-cloud
33:41
clusters are being used today. We saw that they are being used in the recommendation feature or
33:48
the cost arbitrage or cost optimization feature because they want to be able to either use another
33:54
tool that's on a different cloud provider or just be able to switch over to another cloud provider
33:59
all together. We see how multi-cloud helps us as devs because being able to use the tools that we
34:07
want sometimes requires us to reach out to cloud providers that we're not currently on. And it also
34:13
helps us take advantage of all of the regions if we are expected to accommodate customers that are
34:19
all around the world or who may not have as many regions as some other regions that are part of our
34:25
customer base. We also saw how we future-proof, which is the last example I showed you
34:32
specifically dealing with cross-cloud migrations. And then we also saw probably the most common use
34:38
case, which is to take advantage of all of the regions that the three cloud providers offer
34:44
Because with that, instead of being constrained by one single cloud provider and the regions they
34:49
offer, you know how the freedom to choose off of all three of them, which gives us a better
34:55
chance to be able to accommodate all of those customers and to also have higher availability
35:00
and fault tolerance. So thinking about all that, the last thing I want to leave you with
35:08
is this. As you start thinking about applications and as they grow and how we start to make them
35:15
scalable, Brad Lewis from Dell, VP and global leader of Dell, has a really great quote. It says
35:22
If you want to start to have true portability of applications, obviously the data has to go with the application
35:31
And so with that, we're now going to see how quick it actually is to build a multi-cloud cluster
35:38
So let's go here. If I can, let's go. All right. So this is the create a cluster area of MongoDB Atlas
35:53
And you may have seen this before where you choose the cloud provider and region of where you want to host your fully managed database on the cloud, your MongoDB database
36:02
But in this case, we're actually going to build a multi-cloud one. So this little toggle now gives us the option to select some electable nodes which we talk about in a second some read nodes and then the ytics nodes like I mentioned with that recommendation feature So for now we just going to do a fairly simple one
36:22
As you can see, I'm based in Las Vegas, so it automatically shows that because it recognizes that that's the closest region to me
36:30
And so I will leave that there. I'm going to leave that as my highest priority region, meaning this is going to be the one that's prioritized
36:37
and we're going to add two more regions. We're going to add an Azure region
36:43
and we're going to add an AWS region. So for Azure, the next closest one to me
36:49
is probably California. Yes, California. And then we're going to also choose Northern California
36:58
So these are the three regions that are closest to me right now. And as you can see, they're on three different cloud providers
37:03
which is really cool because I haven't seen this anywhere else. So now the next thing we're going to do is we're going to set up the number of nodes
37:11
And so what we'll do here is I'm going to do what's called a 2-2-1 node distribution
37:16
So we'll do 2, 2, and then 1, which gives us a total of 5 electable nodes
37:24
So what this does is allow us to ensure we have continuous read and write availability
37:30
and this is the minimum number that we need to ensure that. And you saw that little warning pop up earlier
37:37
That was just because at the time it calculated, it had an even number of notes
37:43
And you'll see that you get a warning because in order to ensure reliable elections
37:47
AKA when a new primary has to be selected, having an odd number of elections
37:54
make sure that we don't get into a deadlock, that there's an actual primary that wins with one more vote
38:00
So that's why we always want to have an odd number of nodes
38:05
So we'll change that back there. And before we create this cluster, I just want to go really quick
38:09
So electable nodes are the nodes that you use and is most likely used in all those scenarios that I showed you, right
38:16
These are the ones that are in play. These are usually the ones that your production loads
38:21
And these are the ones that are used most of the time for higher availability
38:27
These are the ones that are used in your failover strategies. this is what makes up the the main cluster if you will but if you needed to you can also add
38:36
what's called read only nodes so these are great for let's say you need to service some customers
38:43
and provide data to them super quick they act almost like a cdn but the concept is you open
38:50
or add a node to a place that you want to have faster reads for that local area so i'm in las
38:57
Vegas right now, let's say that we have some customers in Europe. Obviously, if I opened or
39:02
added a read-only node to my cluster in Europe, all the reads can be directed there, which means
39:07
it's much faster. It's lower latency because they don't have to hop all the way over to the US. It
39:12
will go directly to the European node and that will always assure that they have much much faster optimized reads than if they were having to hop over to some place further And you can do that for any kind of region that you need to provide these kinds of fast
39:27
reads for. And lastly, there's something called ytics nodes. So these are very purpose built for the long running queries
39:38
So if you remember the recommendation feature I spoke about where they wanted to add AutoML
39:43
this is exactly what that is used for. Also, if you wanted to host, let's say, some ytics separately, usually what happens is you would have to share that kind of traffic with your production workload, and it would always either slow it down or it would take too long because you would be using the same workloads
40:02
But in this case, you can isolate it on a very specific node and you would keep, that's the
40:08
pretty much the key thing about it is that you can dedicate it for all of your ytics needs
40:12
while making sure all of your production traffic is left alone. It's, it's operating at as efficiently
40:19
as it needs to, and it will stay separated from your ytics nodes or your ytics, ytical
40:27
queries. So that's that. We'll create the cluster, but of course, I've already created one because
40:35
nobody wants to wait for that. So that one is actually being done, but you'll see here
40:40
I have one that's already finished. And there you go. You'll see that the preferred region
40:45
is in GCP Las Vegas. And then you'll see our two secondaries here, which were in Azure California
40:52
And then the third one here, which is a single node in AWS
40:57
And these are all running in the same cluster, which is really, really awesome
41:03
And with all of that running in the same cluster, you still get access to all of the things that you might be used to in MongoDB Atlas
41:10
So you're able to use the key management system that you want
41:15
you're able to have VPC connections, all of the things that you may have already been used to on
41:22
a single region is still available and possible with a multi-cloud cluster
41:32
And with that, if you want to learn more about multi-cloud, because that's just scratching the
41:38
surface, you can check out these links. If you want to set up your own multi-cloud cluster
41:42
I did write a tutorial, a full written tutorial on our developer hub
41:47
And our docs are a really good place to start, as well as the multi-cloud data distribution
41:51
to kind of further delve into multi-cloud as a whole. And with that, Salamat
42:00
Thank you for taking the time to listen to me share with you multi-cloud clusters
42:04
I think they're really, really cool. I think you'll see more of it in the future
42:08
and I hope that you all can understand why it's much more than a buzzword. It is something that
42:14
can be a key part of a solution in the future. So thank you
#Cloud Storage
#Distributed & Cloud Computing
#File Sharing & Hosting
#Networking