Azure Cosmos DB Conference Session 3

0:00
[Music]
0:25
hello
0:25
hello
0:25
hello everybody we are live so uh welcome uh
0:29
everybody we are live so uh welcome uh
0:30
everybody we are live so uh welcome uh to the emir live stream for azure cosmos
0:32
to the emir live stream for azure cosmos
0:32
to the emir live stream for azure cosmos db
0:33
db
0:33
db conf uh the first uh conference for
0:35
conf uh the first uh conference for
0:36
conf uh the first uh conference for cosmos db
0:37
cosmos db
0:37
cosmos db uh so as azure cosmos db conf was
0:39
uh so as azure cosmos db conf was
0:39
uh so as azure cosmos db conf was organized uh to provide
0:41
organized uh to provide
0:41
organized uh to provide a live stage for global cosmos db
0:44
a live stage for global cosmos db
0:44
a live stage for global cosmos db communities
0:45
communities
0:45
communities to share and discover each other so
0:48
to share and discover each other so
0:48
to share and discover each other so over the next three hours you will see
0:51
over the next three hours you will see
0:51
over the next three hours you will see and hear from customers and community
0:53
and hear from customers and community
0:53
and hear from customers and community members
0:54
members
0:54
members who have built some amazing apps and
0:55
who have built some amazing apps and
0:55
who have built some amazing apps and services and they're going to show you
0:57
services and they're going to show you
0:58
services and they're going to show you uh what they've built uh what they're
0:59
uh what they've built uh what they're
1:00
uh what they've built uh what they're learning uh
1:01
learning uh
1:01
learning uh what they learned building on cosmos db
1:03
what they learned building on cosmos db
1:03
what they learned building on cosmos db and for the cloud so
1:04
and for the cloud so
1:04
and for the cloud so you'll hear from partners also who have
1:07
you'll hear from partners also who have
1:07
you'll hear from partners also who have built tools to make
1:08
built tools to make
1:08
built tools to make uh building applications and services
1:10
uh building applications and services
1:10
uh building applications and services with cosmos db
1:12
with cosmos db
1:12
with cosmos db better and easier overall um so i'm theo
1:15
better and easier overall um so i'm theo
1:15
better and easier overall um so i'm theo i'm a pm on the cosmos db team uh i want
1:17
i'm a pm on the cosmos db team uh i want
1:17
i'm a pm on the cosmos db team uh i want to introduce
1:18
to introduce
1:18
to introduce a couple of my colleagues uh girl and
1:20
a couple of my colleagues uh girl and
1:20
a couple of my colleagues uh girl and stephanie they're also pms on the cosmos
1:22
stephanie they're also pms on the cosmos
1:22
stephanie they're also pms on the cosmos db team
1:23
db team
1:23
db team uh girl do you want to take us through
1:25
uh girl do you want to take us through
1:25
uh girl do you want to take us through the the format today
1:27
the the format today
1:27
the the format today sure thanks dear for the introduction hi
1:29
sure thanks dear for the introduction hi
1:29
sure thanks dear for the introduction hi everyone i'm gaul levy and i'm a pm on
1:31
everyone i'm gaul levy and i'm a pm on
1:31
everyone i'm gaul levy and i'm a pm on the cosmos db team
1:32
the cosmos db team
1:32
the cosmos db team in this last live stream for the azure
1:34
in this last live stream for the azure
1:34
in this last live stream for the azure cosmos db conf we'll hear from community
1:36
cosmos db conf we'll hear from community
1:36
cosmos db conf we'll hear from community members from across
1:37
members from across
1:37
members from across the region if you missed any of the live
1:39
the region if you missed any of the live
1:39
the region if you missed any of the live streams
1:40
streams
1:40
streams don't worry you will have these on
1:43
don't worry you will have these on
1:43
don't worry you will have these on demand available shortly
1:45
demand available shortly
1:45
demand available shortly for each of these live sessions over the
1:46
for each of these live sessions over the
1:46
for each of these live sessions over the next three hours we'll have a live q a
1:48
next three hours we'll have a live q a
1:48
next three hours we'll have a live q a link
1:49
link
1:49
link for q a with the speakers if you'd like
1:51
for q a with the speakers if you'd like
1:51
for q a with the speakers if you'd like to ask questions be sure to join
1:53
to ask questions be sure to join
1:53
to ask questions be sure to join the live stream on learn tv
1:56
the live stream on learn tv
1:56
the live stream on learn tv and now a stephanie introducer first
1:58
and now a stephanie introducer first
1:58
and now a stephanie introducer first speaker yes
2:00
speaker yes
2:00
speaker yes thanks everyone thanks call um i'm so
2:02
thanks everyone thanks call um i'm so
2:02
thanks everyone thanks call um i'm so happy to be here so super exciting to be
2:04
happy to be here so super exciting to be
2:04
happy to be here so super exciting to be here
2:04
here
2:04
here um with our first uh conference so first
2:07
um with our first uh conference so first
2:07
um with our first uh conference so first off we're kicking things off with
2:09
off we're kicking things off with
2:09
off we're kicking things off with mikhail shelkoff um he's from the palumi
2:12
mikhail shelkoff um he's from the palumi
2:12
mikhail shelkoff um he's from the palumi team
2:13
team
2:13
team and he's going to be telling us about
2:15
and he's going to be telling us about
2:15
and he's going to be telling us about building modern
2:16
building modern
2:16
building modern infrastructure for modern apps hello
2:19
infrastructure for modern apps hello
2:19
infrastructure for modern apps hello friends
2:19
friends
2:20
friends happy to be here at azure cosmos db
2:22
happy to be here at azure cosmos db
2:22
happy to be here at azure cosmos db conference and
2:23
conference and
2:23
conference and to talk about modern infrastructures
2:25
to talk about modern infrastructures
2:25
to talk about modern infrastructures code and how you can apply to azure
2:27
code and how you can apply to azure
2:27
code and how you can apply to azure cosmos db and other
2:28
cosmos db and other
2:28
cosmos db and other cloud systems that surround it so
2:32
cloud systems that surround it so
2:32
cloud systems that surround it so let's jump to my slides when you use
2:35
let's jump to my slides when you use
2:35
let's jump to my slides when you use cosmos db in your systems you combine it
2:37
cosmos db in your systems you combine it
2:37
cosmos db in your systems you combine it with many other components as well
2:39
with many other components as well
2:39
with many other components as well you develop your applications that are
2:41
you develop your applications that are
2:41
you develop your applications that are deployed to the cloud compute services
2:43
deployed to the cloud compute services
2:43
deployed to the cloud compute services let's say azure kubernetes services in
2:45
let's say azure kubernetes services in
2:45
let's say azure kubernetes services in my demos today
2:47
my demos today
2:47
my demos today maybe those applications are structured
2:49
maybe those applications are structured
2:49
maybe those applications are structured as microservices with all the machinery
2:51
as microservices with all the machinery
2:51
as microservices with all the machinery that
2:52
that
2:52
that you need to run that you use also other
2:55
you need to run that you use also other
2:55
you need to run that you use also other managed services maybe in multiple
2:57
managed services maybe in multiple
2:57
managed services maybe in multiple geographic locations because that's a
2:59
geographic locations because that's a
2:59
geographic locations because that's a strong side of cosmos db
3:01
strong side of cosmos db
3:01
strong side of cosmos db or maybe even across multiple vendors
3:04
or maybe even across multiple vendors
3:04
or maybe even across multiple vendors in any case you need to deploy those
3:06
in any case you need to deploy those
3:06
in any case you need to deploy those applications and infrastructure
3:08
applications and infrastructure
3:08
applications and infrastructure components of course
3:09
components of course
3:09
components of course and today i want to show you how you can
3:11
and today i want to show you how you can
3:11
and today i want to show you how you can use
3:12
use
3:12
use for example c-sharp to provision cloud
3:14
for example c-sharp to provision cloud
3:14
for example c-sharp to provision cloud resources
3:15
resources
3:15
resources and i'm a software developer working an
3:17
and i'm a software developer working an
3:17
and i'm a software developer working an open source tool called polomia
3:19
open source tool called polomia
3:19
open source tool called polomia which allows you exactly this deploy
3:21
which allows you exactly this deploy
3:21
which allows you exactly this deploy azure cosmos db and all the parts of
3:23
azure cosmos db and all the parts of
3:23
azure cosmos db and all the parts of your applications
3:24
your applications
3:24
your applications from net or other programming languages
3:28
from net or other programming languages
3:28
from net or other programming languages so three three cool things about polymer
3:31
so three three cool things about polymer
3:31
so three three cool things about polymer that i want to emphasize today
3:33
that i want to emphasize today
3:33
that i want to emphasize today first is that if you come from a
3:35
first is that if you come from a
3:35
first is that if you come from a developer background
3:36
developer background
3:36
developer background you can use all the skills tools and
3:39
you can use all the skills tools and
3:39
you can use all the skills tools and habits that you already have as a
3:40
habits that you already have as a
3:40
habits that you already have as a developer
3:41
developer
3:41
developer now my demo is going to be in c sharp
3:44
now my demo is going to be in c sharp
3:44
now my demo is going to be in c sharp but that's just because i need to
3:46
but that's just because i need to
3:46
but that's just because i need to need to pick a language for the demo you
3:48
need to pick a language for the demo you
3:48
need to pick a language for the demo you can also use typescript or python to go
3:51
can also use typescript or python to go
3:51
can also use typescript or python to go your favorite language in your own
3:53
your favorite language in your own
3:53
your favorite language in your own projects
3:54
projects
3:54
projects with that i want to show how developers
3:57
with that i want to show how developers
3:57
with that i want to show how developers and operators can
3:58
and operators can
3:58
and operators can need to work together to be productive
4:00
need to work together to be productive
4:00
need to work together to be productive in today's complex
4:02
in today's complex
4:02
in today's complex cloud native world and finally
4:06
cloud native world and finally
4:06
cloud native world and finally with our apis you get access to full api
4:09
with our apis you get access to full api
4:09
with our apis you get access to full api servers of both
4:10
servers of both
4:10
servers of both azure and kubernetes full coverage of
4:13
azure and kubernetes full coverage of
4:13
azure and kubernetes full coverage of all the services and features
4:15
all the services and features
4:15
all the services and features as soon as they are released by
4:16
as soon as they are released by
4:16
as soon as they are released by microsoft or by the kubernetes community
4:19
microsoft or by the kubernetes community
4:19
microsoft or by the kubernetes community so let's get going uh azure has
4:22
so let's get going uh azure has
4:22
so let's get going uh azure has dozens of services and hundreds of
4:24
dozens of services and hundreds of
4:24
dozens of services and hundreds of resource types
4:26
resource types
4:26
resource types and this slide just shows a small subset
4:28
and this slide just shows a small subset
4:28
and this slide just shows a small subset of the of them
4:30
of the of them
4:30
of the of them uh to create all of those of course you
4:32
uh to create all of those of course you
4:32
uh to create all of those of course you can get started with azure portal or
4:34
can get started with azure portal or
4:34
can get started with azure portal or the azure cli these are great things to
4:38
the azure cli these are great things to
4:38
the azure cli these are great things to like explore stuff but they are not
4:40
like explore stuff but they are not
4:40
like explore stuff but they are not great option for production deployment
4:42
great option for production deployment
4:42
great option for production deployment because they are
4:43
because they are
4:43
because they are very manual and hard to reliably
4:46
very manual and hard to reliably
4:46
very manual and hard to reliably reproduce over and over again looking at
4:48
reproduce over and over again looking at
4:48
reproduce over and over again looking at azure cosmos db
4:49
azure cosmos db
4:49
azure cosmos db is like a great example of the breadth
4:52
is like a great example of the breadth
4:52
is like a great example of the breadth of
4:53
of
4:53
of features and management aspects that you
4:55
features and management aspects that you
4:55
features and management aspects that you need to take care of
4:56
need to take care of
4:56
need to take care of you have to manage accounts databases
4:58
you have to manage accounts databases
4:58
you have to manage accounts databases collections apis
5:01
collections apis
5:01
collections apis if you are more of an ito devops person
5:03
if you are more of an ito devops person
5:03
if you are more of an ito devops person you control
5:04
you control
5:04
you control things like replication networking
5:05
things like replication networking
5:06
things like replication networking security backups
5:07
security backups
5:07
security backups scalability monitoring cost
5:10
scalability monitoring cost
5:10
scalability monitoring cost you name it as a developer you are
5:12
you name it as a developer you are
5:12
you name it as a developer you are probably more focused on like
5:14
probably more focused on like
5:14
probably more focused on like fine-tuning your collections with
5:17
fine-tuning your collections with
5:17
fine-tuning your collections with indexes
5:18
indexes
5:18
indexes uh deciding on consistency levels
5:20
uh deciding on consistency levels
5:20
uh deciding on consistency levels managing data flows integrations with
5:22
managing data flows integrations with
5:22
managing data flows integrations with other services
5:24
other services
5:24
other services and so on so it's a bad idea to use ad
5:27
and so on so it's a bad idea to use ad
5:27
and so on so it's a bad idea to use ad hoc manual approach to manage all of
5:29
hoc manual approach to manage all of
5:29
hoc manual approach to manage all of that in in the real
5:30
that in in the real
5:30
that in in the real production applications in real
5:32
production applications in real
5:32
production applications in real organizations
5:34
organizations
5:34
organizations instead you may have heard of
5:35
instead you may have heard of
5:35
instead you may have heard of infrastructure's code where you describe
5:38
infrastructure's code where you describe
5:38
infrastructure's code where you describe your resources your cloud resources in
5:39
your resources your cloud resources in
5:39
your resources your cloud resources in some machine readable format
5:41
some machine readable format
5:41
some machine readable format and then there is a tool that deploys
5:43
and then there is a tool that deploys
5:43
and then there is a tool that deploys the resources for you from your
5:45
the resources for you from your
5:45
the resources for you from your definitions
5:46
definitions
5:46
definitions you may know of arm templates where this
5:48
you may know of arm templates where this
5:48
you may know of arm templates where this machine readable format is json
5:51
machine readable format is json
5:51
machine readable format is json there is now also bicep which introduces
5:53
there is now also bicep which introduces
5:53
there is now also bicep which introduces its own dsl language
5:55
its own dsl language
5:55
its own dsl language in kubernetes that would be yaml files
5:58
in kubernetes that would be yaml files
5:58
in kubernetes that would be yaml files and the helm templates for example
6:01
and the helm templates for example
6:01
and the helm templates for example what we at polymer suggest is that you
6:03
what we at polymer suggest is that you
6:03
what we at polymer suggest is that you modernize your approach and start using
6:05
modernize your approach and start using
6:05
modernize your approach and start using programming languages instead of text
6:06
programming languages instead of text
6:06
programming languages instead of text files
6:07
files
6:07
files the benefits are for example using the
6:10
the benefits are for example using the
6:10
the benefits are for example using the rich tool chain that they are used to
6:12
rich tool chain that they are used to
6:12
rich tool chain that they are used to like compilers ids refactoring and so on
6:15
like compilers ids refactoring and so on
6:15
like compilers ids refactoring and so on that i'm going to show
6:16
that i'm going to show
6:16
that i'm going to show in a demo and then importantly you can
6:18
in a demo and then importantly you can
6:18
in a demo and then importantly you can create reusable abstractions like
6:20
create reusable abstractions like
6:20
create reusable abstractions like functions classes
6:21
functions classes
6:22
functions classes libraries share it with your share them
6:24
libraries share it with your share them
6:24
libraries share it with your share them with your colleagues
6:27
and the larger community as well so
6:32
and the larger community as well so
6:32
and the larger community as well so you can do it since even things like
6:34
you can do it since even things like
6:34
you can do it since even things like unit testing with your code
6:36
unit testing with your code
6:36
unit testing with your code uh for your infrastructure programs as
6:38
uh for your infrastructure programs as
6:38
uh for your infrastructure programs as well
6:39
well
6:39
well how cool is that polymer supports
6:42
how cool is that polymer supports
6:42
how cool is that polymer supports multiple cloud providers aws google
6:44
multiple cloud providers aws google
6:44
multiple cloud providers aws google cloud and others but
6:45
cloud and others but
6:45
cloud and others but our azure story is special because just
6:47
our azure story is special because just
6:47
our azure story is special because just yesterday
6:51
actually we introduced the general
6:52
actually we introduced the general
6:52
actually we introduced the general availability of our
6:54
availability of our
6:54
availability of our native azure provider it supports all of
6:56
native azure provider it supports all of
6:56
native azure provider it supports all of our languages and runtimes
6:58
our languages and runtimes
6:58
our languages and runtimes but even more importantly our sdks are
7:00
but even more importantly our sdks are
7:00
but even more importantly our sdks are automatically generated
7:02
automatically generated
7:02
automatically generated from azure open api specs that microsoft
7:05
from azure open api specs that microsoft
7:05
from azure open api specs that microsoft publishes
7:06
publishes
7:06
publishes that means that you get access to all
7:07
that means that you get access to all
7:07
that means that you get access to all properties of all resources
7:09
properties of all resources
7:09
properties of all resources of all azure services as soon as the
7:11
of all azure services as soon as the
7:11
of all azure services as soon as the specs are publicly released
7:14
specs are publicly released
7:14
specs are publicly released literally on the same day sometimes even
7:16
literally on the same day sometimes even
7:16
literally on the same day sometimes even like before they are released actually
7:18
like before they are released actually
7:18
like before they are released actually i suspect you haven't heard of uh the
7:20
i suspect you haven't heard of uh the
7:20
i suspect you haven't heard of uh the azure web pop sub service
7:23
azure web pop sub service
7:23
azure web pop sub service apparently that's a v-neck service for
7:25
apparently that's a v-neck service for
7:25
apparently that's a v-neck service for signalr which is
7:27
signalr which is
7:27
signalr which is not yet announced as far as i know but
7:29
not yet announced as far as i know but
7:29
not yet announced as far as i know but we already have apis for it
7:30
we already have apis for it
7:30
we already have apis for it because microsoft already published the
7:32
because microsoft already published the
7:32
because microsoft already published the open api specs for the other the other
7:34
open api specs for the other the other
7:34
open api specs for the other the other day
7:36
day
7:36
day and we also have a tool where you can
7:39
and we also have a tool where you can
7:39
and we also have a tool where you can take
7:39
take
7:39
take an arm template and convert it to a
7:41
an arm template and convert it to a
7:41
an arm template and convert it to a draft of your c sharp or typescript
7:43
draft of your c sharp or typescript
7:43
draft of your c sharp or typescript program
7:43
program
7:44
program that you can then adapt to your needs
7:47
that you can then adapt to your needs
7:47
that you can then adapt to your needs with a low effort so here is
7:50
with a low effort so here is
7:50
with a low effort so here is my first example of how you can define
7:52
my first example of how you can define
7:52
my first example of how you can define energy resources with c-sharp in polymer
7:55
energy resources with c-sharp in polymer
7:55
energy resources with c-sharp in polymer each resource is basically constructed
7:56
each resource is basically constructed
7:56
each resource is basically constructed call with a name and a bunch of
7:59
call with a name and a bunch of
7:59
call with a name and a bunch of properties
7:59
properties
7:59
properties in this case i define two cloud
8:01
in this case i define two cloud
8:01
in this case i define two cloud resources a resource group
8:03
resources a resource group
8:03
resources a resource group a container for other resources and the
8:04
a container for other resources and the
8:04
a container for other resources and the store and the cosmos db account
8:08
store and the cosmos db account
8:08
store and the cosmos db account note how i use lots of enums here
8:11
note how i use lots of enums here
8:11
note how i use lots of enums here instead
8:11
instead
8:12
instead of like magic strings that you would use
8:13
of like magic strings that you would use
8:13
of like magic strings that you would use in json and this is all
8:15
in json and this is all
8:15
in json and this is all compile-time checked so if you make a
8:17
compile-time checked so if you make a
8:17
compile-time checked so if you make a typo you will get the compiler and it
8:19
typo you will get the compiler and it
8:19
typo you will get the compiler and it will not
8:20
will not
8:20
will not it will be immediately apparent in the
8:22
it will be immediately apparent in the
8:22
it will be immediately apparent in the id
8:23
id
8:23
id also note how i use the resource group
8:25
also note how i use the resource group
8:25
also note how i use the resource group variable
8:27
variable
8:27
variable in the definition of my database account
8:30
in the definition of my database account
8:30
in the definition of my database account class and that usage by itself creates
8:33
class and that usage by itself creates
8:33
class and that usage by itself creates an automatic dependency between
8:34
an automatic dependency between
8:34
an automatic dependency between resources
8:36
resources
8:36
resources it's not the order statements in my
8:38
it's not the order statements in my
8:38
it's not the order statements in my program it's not imperative because it's
8:39
program it's not imperative because it's
8:39
program it's not imperative because it's declarative
8:40
declarative
8:40
declarative sort of that defines the
8:43
sort of that defines the
8:43
sort of that defines the deployment order it's those dependencies
8:46
deployment order it's those dependencies
8:46
deployment order it's those dependencies through the variables
8:47
through the variables
8:47
through the variables that define the actual deployment
8:50
that define the actual deployment
8:50
that define the actual deployment sequence now a lot of companies
8:54
sequence now a lot of companies
8:54
sequence now a lot of companies these days also use kubernetes as their
8:56
these days also use kubernetes as their
8:56
these days also use kubernetes as their application platform
8:57
application platform
8:57
application platform with pollumi it's the same story for
8:59
with pollumi it's the same story for
8:59
with pollumi it's the same story for kubernetes as for
9:00
kubernetes as for
9:00
kubernetes as for azure you can manage hundred percent of
9:02
azure you can manage hundred percent of
9:02
azure you can manage hundred percent of your api of api surface of kubernetes
9:05
your api of api surface of kubernetes
9:05
your api of api surface of kubernetes and you can deploy actually both azure
9:07
and you can deploy actually both azure
9:07
and you can deploy actually both azure and kubernetes from the same program
9:09
and kubernetes from the same program
9:09
and kubernetes from the same program as i'm going to show in the demo and all
9:12
as i'm going to show in the demo and all
9:12
as i'm going to show in the demo and all with the safety and pleasure
9:14
with the safety and pleasure
9:14
with the safety and pleasure of working with uh net or your favorite
9:17
of working with uh net or your favorite
9:17
of working with uh net or your favorite uh
9:17
uh
9:17
uh runtime so actually these are all my
9:20
runtime so actually these are all my
9:20
runtime so actually these are all my slides for today and the rest is going
9:21
slides for today and the rest is going
9:21
slides for today and the rest is going to be
9:22
to be
9:22
to be my demo
9:25
in the demo i'm going to use dot net 5
9:28
in the demo i'm going to use dot net 5
9:28
in the demo i'm going to use dot net 5 and c sharp 9
9:29
and c sharp 9
9:29
and c sharp 9 to define a polymer program that
9:30
to define a polymer program that
9:30
to define a polymer program that provisions an aks cluster a cosmos db
9:33
provisions an aks cluster a cosmos db
9:33
provisions an aks cluster a cosmos db account
9:34
account
9:34
account with mongodb api then it deploys a to-do
9:37
with mongodb api then it deploys a to-do
9:37
with mongodb api then it deploys a to-do app
9:38
app
9:38
app with a helm chart uh that uses that
9:41
with a helm chart uh that uses that
9:41
with a helm chart uh that uses that cosmos db as a data storage
9:43
cosmos db as a data storage
9:44
cosmos db as a data storage uh it's a it's a to-do app obviously
9:46
uh it's a it's a to-do app obviously
9:46
uh it's a it's a to-do app obviously it's very simplistic version of
9:47
it's very simplistic version of
9:47
it's very simplistic version of real-world applications
9:49
real-world applications
9:49
real-world applications but as you can see there is already a
9:51
but as you can see there is already a
9:51
but as you can see there is already a bunch of moving parts that i have to
9:52
bunch of moving parts that i have to
9:52
bunch of moving parts that i have to manage and integrate
9:53
manage and integrate
9:54
manage and integrate correctly so i'm switching over to my
9:57
correctly so i'm switching over to my
9:57
correctly so i'm switching over to my visual studio where i have a
10:03
i hope you can see it yes you can see it
10:06
i hope you can see it yes you can see it
10:06
i hope you can see it yes you can see it now
10:06
now
10:06
now so i'm on my visual studio where i have
10:09
so i'm on my visual studio where i have
10:09
so i'm on my visual studio where i have a c-sharp project bootstrapped and
10:11
a c-sharp project bootstrapped and
10:11
a c-sharp project bootstrapped and that's a
10:11
that's a
10:11
that's a normal.net 5 project and executable
10:15
normal.net 5 project and executable
10:15
normal.net 5 project and executable program and you can see that they have a
10:18
program and you can see that they have a
10:18
program and you can see that they have a bunch of nuget references they are all
10:20
bunch of nuget references they are all
10:20
bunch of nuget references they are all from polymer different
10:21
from polymer different
10:21
from polymer different providers of polymer one can azure
10:23
providers of polymer one can azure
10:24
providers of polymer one can azure manage azure resources one can manage
10:26
manage azure resources one can manage
10:26
manage azure resources one can manage communities resources something to
10:28
communities resources something to
10:28
communities resources something to create azure id accounts
10:30
create azure id accounts
10:30
create azure id accounts uh for example as a selkies that i need
10:33
uh for example as a selkies that i need
10:33
uh for example as a selkies that i need to create a
10:34
to create a
10:34
to create a kubernetes cluster and so on and so
10:36
kubernetes cluster and so on and so
10:36
kubernetes cluster and so on and so forth so my program
10:38
forth so my program
10:38
forth so my program is empty at the moment and i have a
10:40
is empty at the moment and i have a
10:40
is empty at the moment and i have a couple of snippets to
10:42
couple of snippets to
10:42
couple of snippets to avoid typing and sort of get faster go
10:45
avoid typing and sort of get faster go
10:45
avoid typing and sort of get faster go get going faster my first at first i
10:49
get going faster my first at first i
10:49
get going faster my first at first i define my programmer i use a bunch of
10:51
define my programmer i use a bunch of
10:52
define my programmer i use a bunch of using statements to get all the packages
10:53
using statements to get all the packages
10:53
using statements to get all the packages right
10:54
right
10:54
right you can see i'm going to create a bunch
10:55
you can see i'm going to create a bunch
10:55
you can see i'm going to create a bunch of kubernetes objects
10:57
of kubernetes objects
10:57
of kubernetes objects and then i'm using the top level
10:59
and then i'm using the top level
10:59
and then i'm using the top level statements as my
11:01
statements as my
11:01
statements as my console application code and all the
11:03
console application code and all the
11:03
console application code and all the program just consists of one
11:05
program just consists of one
11:05
program just consists of one line basically it's saying follow me
11:08
line basically it's saying follow me
11:08
line basically it's saying follow me deployment run async and please deploy
11:10
deployment run async and please deploy
11:10
deployment run async and please deploy the stack that i define below and stack
11:13
the stack that i define below and stack
11:13
the stack that i define below and stack is a class
11:13
is a class
11:14
is a class which derives from a base class stack
11:16
which derives from a base class stack
11:16
which derives from a base class stack from aluminum space
11:18
from aluminum space
11:18
from aluminum space and this is the constructor and i need
11:20
and this is the constructor and i need
11:20
and this is the constructor and i need to define my azure resources in here
11:23
to define my azure resources in here
11:23
to define my azure resources in here so my first azure resource is going to
11:25
so my first azure resource is going to
11:25
so my first azure resource is going to be a resource group of course
11:27
be a resource group of course
11:27
be a resource group of course because i needed to produce another
11:29
because i needed to produce another
11:29
because i needed to produce another resources and that's all i need to
11:31
resources and that's all i need to
11:31
resources and that's all i need to define a resource group i just give it a
11:33
define a resource group i just give it a
11:33
define a resource group i just give it a name
11:33
name
11:33
name i have a config file already on my
11:35
i have a config file already on my
11:35
i have a config file already on my machine that says that my default
11:37
machine that says that my default
11:37
machine that says that my default location is west europe
11:38
location is west europe
11:38
location is west europe where i reside so that's all i need to
11:41
where i reside so that's all i need to
11:41
where i reside so that's all i need to start deploying
11:42
start deploying
11:42
start deploying my application and actually i can now
11:45
my application and actually i can now
11:45
my application and actually i can now switch over
11:47
switch over
11:47
switch over to powershell console
11:51
to powershell console
11:51
to powershell console i'm on the folder with that project that
11:53
i'm on the folder with that project that
11:54
i'm on the folder with that project that i just created
11:55
i just created
11:55
i just created and now to deploy this code i need to
11:58
and now to deploy this code i need to
11:58
and now to deploy this code i need to type pull me up
12:00
type pull me up
12:00
type pull me up in my console maybe i will make it a bit
12:02
in my console maybe i will make it a bit
12:02
in my console maybe i will make it a bit bigger
12:04
bigger
12:04
bigger and then hit enter and then blooming
12:07
and then hit enter and then blooming
12:07
and then hit enter and then blooming runs
12:08
runs
12:08
runs it compiles the program that i just
12:10
it compiles the program that i just
12:10
it compiles the program that i just created in visual studio
12:12
created in visual studio
12:12
created in visual studio and then it will run it as a console
12:14
and then it will run it as a console
12:14
and then it will run it as a console application and it will get the
12:16
application and it will get the
12:16
application and it will get the definitions of
12:18
definitions of
12:18
definitions of resources that i have there and will
12:20
resources that i have there and will
12:20
resources that i have there and will show me the preview so nothing is being
12:22
show me the preview so nothing is being
12:22
show me the preview so nothing is being deployed yet
12:23
deployed yet
12:23
deployed yet it's just oh okay so i don't have
12:27
it's just oh okay so i don't have
12:27
it's just oh okay so i don't have that's a good thing i don't have
12:28
that's a good thing i don't have
12:28
that's a good thing i don't have location configured in my uh
12:31
location configured in my uh
12:32
location configured in my uh config file so i need to run another
12:33
config file so i need to run another
12:33
config file so i need to run another limit program that will say
12:36
limit program that will say
12:36
limit program that will say that i'm actually in west europe so you
12:38
that i'm actually in west europe so you
12:38
that i'm actually in west europe so you can see how it's done
12:40
can see how it's done
12:40
can see how it's done okay and now invest europe let's try it
12:43
okay and now invest europe let's try it
12:43
okay and now invest europe let's try it again
12:45
again
12:46
again pull me up again it compiles again runs
12:48
pull me up again it compiles again runs
12:48
pull me up again it compiles again runs my code and will again
12:50
my code and will again
12:50
my code and will again this time it should show me the preview
12:52
this time it should show me the preview
12:52
this time it should show me the preview of which resources are about to be
12:54
of which resources are about to be
12:54
of which resources are about to be created in in azure
12:57
created in in azure
12:57
created in in azure yes i can see that uh it wants to create
12:59
yes i can see that uh it wants to create
12:59
yes i can see that uh it wants to create sort of two things one is it just a
13:01
sort of two things one is it just a
13:01
sort of two things one is it just a stack
13:02
stack
13:02
stack that is like a high level container of
13:04
that is like a high level container of
13:04
that is like a high level container of my resources it doesn't create anything
13:06
my resources it doesn't create anything
13:06
my resources it doesn't create anything in azure
13:07
in azure
13:07
in azure and then the second one is a resource
13:08
and then the second one is a resource
13:08
and then the second one is a resource group this looks good to me so i hit yes
13:11
group this looks good to me so i hit yes
13:11
group this looks good to me so i hit yes and this time my program runs again and
13:14
and this time my program runs again and
13:14
and this time my program runs again and the
13:14
the
13:14
the polum will go ahead and actually
13:16
polum will go ahead and actually
13:16
polum will go ahead and actually provision resources in the cloud for me
13:19
provision resources in the cloud for me
13:19
provision resources in the cloud for me so it should create a stack which is a
13:21
so it should create a stack which is a
13:21
so it should create a stack which is a no opens in terms of azure
13:23
no opens in terms of azure
13:23
no opens in terms of azure and then it creates a resource group and
13:25
and then it creates a resource group and
13:25
and then it creates a resource group and it took me about nine seconds to do all
13:27
it took me about nine seconds to do all
13:27
it took me about nine seconds to do all that
13:28
that
13:28
that okay this looks good switching back to
13:30
okay this looks good switching back to
13:30
okay this looks good switching back to my visual studio
13:32
my visual studio
13:32
my visual studio uh after research group i want to
13:34
uh after research group i want to
13:34
uh after research group i want to provision my cosmos db of course
13:38
provision my cosmos db of course
13:38
provision my cosmos db of course so i have snippet that creates
13:42
so i have snippet that creates
13:42
so i have snippet that creates a cosmos db mongodb database
13:45
a cosmos db mongodb database
13:45
a cosmos db mongodb database because that's what my application is
13:47
because that's what my application is
13:47
because that's what my application is going to expect
13:49
going to expect
13:49
going to expect it expects them a mongodb actually
13:51
it expects them a mongodb actually
13:51
it expects them a mongodb actually doesn't know that it's going to be
13:52
doesn't know that it's going to be
13:52
doesn't know that it's going to be cosmos
13:52
cosmos
13:52
cosmos so just uh using a mongodb api for
13:55
so just uh using a mongodb api for
13:55
so just uh using a mongodb api for cosmos
13:56
cosmos
13:56
cosmos very powerful feature and then i can go
14:00
very powerful feature and then i can go
14:00
very powerful feature and then i can go proceed let's deploy my application okay
14:03
proceed let's deploy my application okay
14:03
proceed let's deploy my application okay so i
14:03
so i
14:03
so i defined a new resource and actually
14:06
defined a new resource and actually
14:06
defined a new resource and actually it's not one resource it's two resources
14:08
it's not one resource it's two resources
14:08
it's not one resource it's two resources and i have a helper component here
14:11
and i have a helper component here
14:11
and i have a helper component here in my program uh it's not a class and
14:14
in my program uh it's not a class and
14:14
in my program uh it's not a class and it's a good
14:14
it's a good
14:14
it's a good demonstration of how you can create
14:16
demonstration of how you can create
14:16
demonstration of how you can create abstractions in your own code
14:18
abstractions in your own code
14:18
abstractions in your own code so i created a class which uh does
14:22
so i created a class which uh does
14:22
so i created a class which uh does an integrity details of provision in a
14:24
an integrity details of provision in a
14:24
an integrity details of provision in a cosmos db account
14:25
cosmos db account
14:26
cosmos db account with all like offer types and the kind
14:28
with all like offer types and the kind
14:28
with all like offer types and the kind of
14:29
of
14:29
of mongodb defines the consistency layer
14:32
mongodb defines the consistency layer
14:32
mongodb defines the consistency layer and so on and so forth locations where i
14:33
and so on and so forth locations where i
14:33
and so on and so forth locations where i want to deploy
14:35
want to deploy
14:35
want to deploy and then the second resource is the
14:37
and then the second resource is the
14:37
and then the second resource is the database
14:39
database
14:39
database that i need to create for my to-do app
14:41
that i need to create for my to-do app
14:41
that i need to create for my to-do app and it's called
14:42
and it's called
14:42
and it's called to-do's of course so now when i edit
14:46
to-do's of course so now when i edit
14:46
to-do's of course so now when i edit this
14:47
this
14:48
this component call to my program my program
14:50
component call to my program my program
14:50
component call to my program my program still looks very simple so i can
14:51
still looks very simple so i can
14:51
still looks very simple so i can reuse this class over and over again
14:55
reuse this class over and over again
14:55
reuse this class over and over again i can switch back to the cli and run
14:57
i can switch back to the cli and run
14:57
i can switch back to the cli and run follow me up again
15:00
follow me up again
15:00
follow me up again and this time the same happens it runs
15:02
and this time the same happens it runs
15:02
and this time the same happens it runs my program
15:03
my program
15:03
my program it will see that now i define two
15:05
it will see that now i define two
15:05
it will see that now i define two resources one is resource group
15:07
resources one is resource group
15:07
resources one is resource group actually three resources a resource
15:08
actually three resources a resource
15:08
actually three resources a resource group a cosmos db account
15:11
group a cosmos db account
15:11
group a cosmos db account and the database and it compares that
15:13
and the database and it compares that
15:13
and the database and it compares that definition with the previous deployment
15:16
definition with the previous deployment
15:16
definition with the previous deployment where i already have resource group
15:18
where i already have resource group
15:18
where i already have resource group deployed to azure
15:19
deployed to azure
15:19
deployed to azure it now shows me the difference between
15:21
it now shows me the difference between
15:21
it now shows me the difference between what i have in azure and what i have
15:23
what i have in azure and what i have
15:23
what i have in azure and what i have in my program and the difference is that
15:25
in my program and the difference is that
15:26
in my program and the difference is that it's going to create
15:26
it's going to create
15:26
it's going to create a database account and mongodb database
15:30
a database account and mongodb database
15:30
a database account and mongodb database and it will not touch my resource group
15:31
and it will not touch my resource group
15:32
and it will not touch my resource group because because it's already there
15:33
because because it's already there
15:34
because because it's already there so i don't have to write like i will hit
15:37
so i don't have to write like i will hit
15:37
so i don't have to write like i will hit yes to start the deployment
15:39
yes to start the deployment
15:39
yes to start the deployment uh i don't have to write like migration
15:41
uh i don't have to write like migration
15:41
uh i don't have to write like migration scripts from state a to state b
15:43
scripts from state a to state b
15:43
scripts from state a to state b i just keep adding more and more
15:44
i just keep adding more and more
15:44
i just keep adding more and more resources to my program and then polymer
15:46
resources to my program and then polymer
15:46
resources to my program and then polymer takes care of the state and comparing
15:48
takes care of the state and comparing
15:48
takes care of the state and comparing the previous state with the next state
15:49
the previous state with the next state
15:49
the previous state with the next state and figure out
15:50
and figure out
15:50
and figure out out the best path forward and that's a
15:53
out the best path forward and that's a
15:53
out the best path forward and that's a great
15:54
great
15:54
great difference with for example using azure
15:56
difference with for example using azure
15:56
difference with for example using azure sdk directory where you would have to
15:59
sdk directory where you would have to
15:59
sdk directory where you would have to like create an imperative step-by-step
16:01
like create an imperative step-by-step
16:01
like create an imperative step-by-step program of what exactly to be
16:03
program of what exactly to be
16:03
program of what exactly to be provisioned
16:04
provisioned
16:04
provisioned this time or versus like upgrading from
16:07
this time or versus like upgrading from
16:07
this time or versus like upgrading from version a
16:08
version a
16:08
version a to version b while in polymer you define
16:10
to version b while in polymer you define
16:10
to version b while in polymer you define desired state configuration
16:12
desired state configuration
16:12
desired state configuration where you keep adding more and more
16:13
where you keep adding more and more
16:13
where you keep adding more and more resources and then the state is taken
16:15
resources and then the state is taken
16:15
resources and then the state is taken care of by the client
16:18
care of by the client
16:18
care of by the client so actually as you may know progressing
16:20
so actually as you may know progressing
16:20
so actually as you may know progressing the database account takes
16:21
the database account takes
16:21
the database account takes about 5-10 minutes i don't know so i'll
16:24
about 5-10 minutes i don't know so i'll
16:24
about 5-10 minutes i don't know so i'll leave it running here and we'll go back
16:26
leave it running here and we'll go back
16:26
leave it running here and we'll go back to visual studio to
16:27
to visual studio to
16:27
to visual studio to complete my project so next thing
16:31
complete my project so next thing
16:31
complete my project so next thing that here is a
16:35
that here is a
16:36
that here is a case cluster i have another component
16:39
case cluster i have another component
16:39
case cluster i have another component that
16:39
that
16:39
that abstracts away some security details of
16:42
abstracts away some security details of
16:42
abstracts away some security details of producing a kubernetes cluster for
16:44
producing a kubernetes cluster for
16:44
producing a kubernetes cluster for azure kubernetes servers it needs like
16:47
azure kubernetes servers it needs like
16:47
azure kubernetes servers it needs like ssl keys
16:49
ssl keys
16:49
ssl keys applications some other
16:53
applications some other
16:53
applications some other resources and of course the managed
16:54
resources and of course the managed
16:54
resources and of course the managed cluster itself but it's all hidden in
16:57
cluster itself but it's all hidden in
16:57
cluster itself but it's all hidden in the
16:57
the
16:57
the this component that i created and i just
16:59
this component that i created and i just
16:59
this component that i created and i just need to pass a resource group
17:01
need to pass a resource group
17:01
need to pass a resource group kubernetes version that i want not count
17:02
kubernetes version that i want not count
17:02
kubernetes version that i want not count the node size
17:04
the node size
17:04
the node size for my pool that's about it and now i
17:07
for my pool that's about it and now i
17:07
for my pool that's about it and now i can start
17:08
can start
17:08
can start provisioning resources to the same
17:11
provisioning resources to the same
17:11
provisioning resources to the same to this aks cluster from the same
17:14
to this aks cluster from the same
17:14
to this aks cluster from the same program
17:15
program
17:15
program one of the outputs of that component if
17:17
one of the outputs of that component if
17:17
one of the outputs of that component if i type here
17:18
i type here
17:18
i type here you can see that i'm i can use an
17:19
you can see that i'm i can use an
17:20
you can see that i'm i can use an intellisense and everything
17:21
intellisense and everything
17:21
intellisense and everything it's going to be a coupon cube config
17:24
it's going to be a coupon cube config
17:24
it's going to be a coupon cube config and that's the
17:25
and that's the
17:25
and that's the config that's going to be calculated
17:26
config that's going to be calculated
17:26
config that's going to be calculated after the cluster is deployed and i can
17:28
after the cluster is deployed and i can
17:28
after the cluster is deployed and i can use that keep config
17:29
use that keep config
17:30
use that keep config immediately even before the cluster is
17:33
immediately even before the cluster is
17:33
immediately even before the cluster is deployed
17:34
deployed
17:34
deployed to start creating kubernetes resources
17:36
to start creating kubernetes resources
17:36
to start creating kubernetes resources here for example
17:38
here for example
17:38
here for example i need to put a connection stream
17:40
i need to put a connection stream
17:40
i need to put a connection stream somewhere so i create a kubernetes
17:41
somewhere so i create a kubernetes
17:41
somewhere so i create a kubernetes secret
17:42
secret
17:42
secret that contains a string of my connections
17:44
that contains a string of my connections
17:44
that contains a string of my connections into the cosmos db
17:45
into the cosmos db
17:45
into the cosmos db mongodb api and then
17:49
mongodb api and then
17:49
mongodb api and then you can see i'm using a function here c
17:51
you can see i'm using a function here c
17:51
you can see i'm using a function here c sharp function to format that
17:52
sharp function to format that
17:52
sharp function to format that connection string and that's another
17:54
connection string and that's another
17:54
connection string and that's another benefit of using c sharp or
17:56
benefit of using c sharp or
17:56
benefit of using c sharp or other general programming language as
17:58
other general programming language as
17:58
other general programming language as opposed to json
17:59
opposed to json
18:00
opposed to json you can just write a simple function or
18:02
you can just write a simple function or
18:02
you can just write a simple function or class
18:03
class
18:03
class with methods as you would do in your
18:05
with methods as you would do in your
18:06
with methods as you would do in your application and do whatever you want
18:07
application and do whatever you want
18:07
application and do whatever you want there
18:07
there
18:08
there format strings or load something from
18:10
format strings or load something from
18:10
format strings or load something from external configuration file
18:12
external configuration file
18:12
external configuration file or make a call to the external api if
18:14
or make a call to the external api if
18:14
or make a call to the external api if you want and like format
18:16
you want and like format
18:16
you want and like format the thing that you need to deploy as a
18:17
the thing that you need to deploy as a
18:18
the thing that you need to deploy as a secret in kubernetes and just assign it
18:19
secret in kubernetes and just assign it
18:19
secret in kubernetes and just assign it to a property
18:22
to a property
18:22
to a property and then the next step is going to be
18:25
and then the next step is going to be
18:25
and then the next step is going to be deploying a health chart
18:29
deploying a health chart
18:29
deploying a health chart there is a health chart just a demo helm
18:31
there is a health chart just a demo helm
18:31
there is a health chart just a demo helm chart by bitnami
18:32
chart by bitnami
18:32
chart by bitnami that deploys it to the app given a
18:34
that deploys it to the app given a
18:34
that deploys it to the app given a mongodb connection string
18:36
mongodb connection string
18:36
mongodb connection string in this case i say that hey it needs a
18:39
in this case i say that hey it needs a
18:39
in this case i say that hey it needs a very cold external db and i will point
18:41
very cold external db and i will point
18:41
very cold external db and i will point it to the secret that i created on the
18:43
it to the secret that i created on the
18:43
it to the secret that i created on the previous step
18:44
previous step
18:44
previous step and it will read this secret value at
18:47
and it will read this secret value at
18:47
and it will read this secret value at runtime from the kubernetes
18:48
runtime from the kubernetes
18:48
runtime from the kubernetes when it runs in kubernetes and will
18:50
when it runs in kubernetes and will
18:50
when it runs in kubernetes and will create a load balancer in front of it
18:52
create a load balancer in front of it
18:52
create a load balancer in front of it and it's going to be my application
18:54
and it's going to be my application
18:54
and it's going to be my application instead of that you could deploy your
18:56
instead of that you could deploy your
18:56
instead of that you could deploy your asp.net app
18:57
asp.net app
18:57
asp.net app or whatever kind of application you
19:00
or whatever kind of application you
19:00
or whatever kind of application you create to connect to cosmos db
19:01
create to connect to cosmos db
19:01
create to connect to cosmos db this is just very easy to demo
19:05
this is just very easy to demo
19:05
this is just very easy to demo and then the final step in my program
19:06
and then the final step in my program
19:06
and then the final step in my program i'm almost done
19:08
i'm almost done
19:08
i'm almost done is to export the url of the resulting
19:12
is to export the url of the resulting
19:12
is to export the url of the resulting application of course like i can get it
19:14
application of course like i can get it
19:14
application of course like i can get it from
19:15
from
19:15
from a service object and kubernetes take its
19:17
a service object and kubernetes take its
19:17
a service object and kubernetes take its load balancer ingress ip
19:20
load balancer ingress ip
19:20
load balancer ingress ip all of that with c-sharp again all with
19:22
all of that with c-sharp again all with
19:22
all of that with c-sharp again all with the types and intellisense if i make
19:24
the types and intellisense if i make
19:24
the types and intellisense if i make like a typo here it will complain that
19:27
like a typo here it will complain that
19:27
like a typo here it will complain that there is no property into grass
19:29
there is no property into grass
19:29
there is no property into grass and so i get a very fast feedback loop
19:32
and so i get a very fast feedback loop
19:32
and so i get a very fast feedback loop while i'm developing code for this and
19:34
while i'm developing code for this and
19:34
while i'm developing code for this and i'm
19:35
i'm
19:35
i'm can be confident that it sort of works
19:38
can be confident that it sort of works
19:38
can be confident that it sort of works before i even started deploying
19:41
before i even started deploying
19:41
before i even started deploying and then i assigned that ip plus and
19:44
and then i assigned that ip plus and
19:44
and then i assigned that ip plus and http
19:45
http
19:45
http to the endpoint output and that's how i
19:47
to the endpoint output and that's how i
19:47
to the endpoint output and that's how i define the output of my program so when
19:49
define the output of my program so when
19:49
define the output of my program so when my
19:49
my
19:49
my deployment is done it will print out the
19:52
deployment is done it will print out the
19:52
deployment is done it will print out the end point
19:54
end point
19:54
end point at the end of the deployment and i will
19:56
at the end of the deployment and i will
19:56
at the end of the deployment and i will be able to use it
19:57
be able to use it
19:57
be able to use it immediately after so that's it that's a
20:00
immediately after so that's it that's a
20:00
immediately after so that's it that's a complete program i'll just hit save
20:03
complete program i'll just hit save
20:03
complete program i'll just hit save and switch over to the cli let's see if
20:05
and switch over to the cli let's see if
20:05
and switch over to the cli let's see if yes cosmos db
20:06
yes cosmos db
20:06
yes cosmos db actually present i was like it got
20:09
actually present i was like it got
20:09
actually present i was like it got created in four minutes so i can
20:13
created in four minutes so i can
20:13
created in four minutes so i can run polymer up again now it compares the
20:16
run polymer up again now it compares the
20:16
run polymer up again now it compares the state the previous state
20:18
state the previous state
20:18
state the previous state of my deployment with the database
20:20
of my deployment with the database
20:20
of my deployment with the database versus the current uh
20:22
versus the current uh
20:22
versus the current uh program and it should figure out that it
20:24
program and it should figure out that it
20:24
program and it should figure out that it should deploy an aks cluster
20:26
should deploy an aks cluster
20:26
should deploy an aks cluster and a bunch of kubernetes objects to it
20:30
and a bunch of kubernetes objects to it
20:30
and a bunch of kubernetes objects to it let's see what the previous shows real
20:32
let's see what the previous shows real
20:32
let's see what the previous shows real quick you can already see
20:34
quick you can already see
20:34
quick you can already see it it computes yeah okay i'll scroll up
20:36
it it computes yeah okay i'll scroll up
20:36
it it computes yeah okay i'll scroll up a bit
20:37
a bit
20:37
a bit so it has like a aks cluster
20:40
so it has like a aks cluster
20:40
so it has like a aks cluster with a managed cluster in container
20:43
with a managed cluster in container
20:43
with a managed cluster in container servers
20:44
servers
20:44
servers also it has a helm chart that i defined
20:46
also it has a helm chart that i defined
20:46
also it has a helm chart that i defined and the secret that i defined with
20:48
and the secret that i defined with
20:48
and the secret that i defined with connection scenes so this looks good
20:50
connection scenes so this looks good
20:50
connection scenes so this looks good i'll hit yes and i don't have time to
20:52
i'll hit yes and i don't have time to
20:52
i'll hit yes and i don't have time to wait for aks
20:53
wait for aks
20:53
wait for aks so i'll just switch over to next
20:56
so i'll just switch over to next
20:56
so i'll just switch over to next tab in my powershell where i had the
20:59
tab in my powershell where i had the
20:59
tab in my powershell where i had the same
21:00
same
21:00
same deployment but already done and finished
21:02
deployment but already done and finished
21:02
deployment but already done and finished it will take about
21:03
it will take about
21:03
it will take about three four minutes more so in the end
21:06
three four minutes more so in the end
21:06
three four minutes more so in the end all this is created and you can see that
21:08
all this is created and you can see that
21:08
all this is created and you can see that the output that i see here
21:09
the output that i see here
21:09
the output that i see here is the endpoint to my application and if
21:12
is the endpoint to my application and if
21:12
is the endpoint to my application and if i switch over to
21:13
i switch over to
21:13
i switch over to the browser this is there to do that we
21:15
the browser this is there to do that we
21:15
the browser this is there to do that we deployed
21:16
deployed
21:16
deployed and then like if i do something at here
21:21
and then like if i do something at here
21:21
and then like if i do something at here this is start already in cosmos db that
21:23
this is start already in cosmos db that
21:23
this is start already in cosmos db that i just provisioned with my c sharp code
21:26
i just provisioned with my c sharp code
21:26
i just provisioned with my c sharp code all right so quickly back to the slides
21:32
all right so quickly back to the slides
21:32
all right so quickly back to the slides uh so that was the case where i deployed
21:35
uh so that was the case where i deployed
21:35
uh so that was the case where i deployed all the
21:35
all the
21:35
all the components from the same application
21:37
components from the same application
21:37
components from the same application from the same program as
21:38
from the same program as
21:38
from the same program as in c sharp in practice it's not likely
21:41
in c sharp in practice it's not likely
21:41
in c sharp in practice it's not likely that the same person in your
21:42
that the same person in your
21:42
that the same person in your organization is going to deploy all
21:44
organization is going to deploy all
21:44
organization is going to deploy all those things
21:45
those things
21:45
those things at the same time so what we see more
21:49
at the same time so what we see more
21:49
at the same time so what we see more in the biology companies or even in
21:51
in the biology companies or even in
21:51
in the biology companies or even in smaller companies with multiple teams
21:53
smaller companies with multiple teams
21:53
smaller companies with multiple teams that they have some specialization there
21:55
that they have some specialization there
21:55
that they have some specialization there some people
21:57
some people
21:57
some people work on like underlying things like
22:00
work on like underlying things like
22:00
work on like underlying things like kubernetes clusters but also security
22:01
kubernetes clusters but also security
22:02
kubernetes clusters but also security networking and storage
22:03
networking and storage
22:03
networking and storage monitoring scalability multiple
22:06
monitoring scalability multiple
22:06
monitoring scalability multiple environments
22:07
environments
22:07
environments creating guard delays for for their
22:08
creating guard delays for for their
22:08
creating guard delays for for their developers
22:10
developers
22:10
developers these are like platform teams that are
22:12
these are like platform teams that are
22:12
these are like platform teams that are working on the internal platforms for
22:14
working on the internal platforms for
22:14
working on the internal platforms for their applications
22:15
their applications
22:15
their applications in the company to be deployed and then
22:17
in the company to be deployed and then
22:17
in the company to be deployed and then application developers they usually
22:19
application developers they usually
22:19
application developers they usually don't really want to care about all
22:20
don't really want to care about all
22:20
don't really want to care about all those
22:21
those
22:21
those details they care about the application
22:23
details they care about the application
22:23
details they care about the application code the data
22:25
code the data
22:25
code the data that they manage like messaging login
22:27
that they manage like messaging login
22:27
that they manage like messaging login performance and integrations with
22:28
performance and integrations with
22:28
performance and integrations with different services internally or
22:30
different services internally or
22:30
different services internally or externally
22:31
externally
22:31
externally so there are multiple roles and what how
22:34
so there are multiple roles and what how
22:34
so there are multiple roles and what how this translates
22:35
this translates
22:35
this translates uh in pollumi is that probably a
22:37
uh in pollumi is that probably a
22:37
uh in pollumi is that probably a platform team will create
22:39
platform team will create
22:39
platform team will create a stack or multiple stacks where they
22:41
a stack or multiple stacks where they
22:41
a stack or multiple stacks where they define all those level
22:43
define all those level
22:43
define all those level low level details and they will deploy
22:45
low level details and they will deploy
22:45
low level details and they will deploy them
22:46
them
22:46
them on their cadence when they want from and
22:48
on their cadence when they want from and
22:48
on their cadence when they want from and with their permissions and their
22:49
with their permissions and their
22:50
with their permissions and their credentials
22:51
credentials
22:51
credentials and then they will share something like
22:54
and then they will share something like
22:54
and then they will share something like cube configs and other connection
22:56
cube configs and other connection
22:56
cube configs and other connection streams with
22:58
streams with
22:58
streams with application developers and then they can
23:00
application developers and then they can
23:00
application developers and then they can import it and sort of
23:01
import it and sort of
23:01
import it and sort of based on that deploy their kubernetes
23:04
based on that deploy their kubernetes
23:04
based on that deploy their kubernetes services deployments and so on
23:06
services deployments and so on
23:06
services deployments and so on to the clusters that were already
23:07
to the clusters that were already
23:07
to the clusters that were already provisioned and ready to go
23:09
provisioned and ready to go
23:09
provisioned and ready to go and even um with polymer platform teams
23:12
and even um with polymer platform teams
23:12
and even um with polymer platform teams can create like
23:13
can create like
23:13
can create like the components like i showed you where
23:16
the components like i showed you where
23:16
the components like i showed you where they
23:16
they
23:16
they create the blast configurations of
23:19
create the blast configurations of
23:19
create the blast configurations of application deployments for example
23:21
application deployments for example
23:21
application deployments for example and then they can share it as a nuget
23:22
and then they can share it as a nuget
23:22
and then they can share it as a nuget package internally
23:24
package internally
23:24
package internally or to the community and the application
23:27
or to the community and the application
23:27
or to the community and the application teams will be able to
23:28
teams will be able to
23:28
teams will be able to get started quickly and
23:31
get started quickly and
23:31
get started quickly and use those in their applications uh
23:35
use those in their applications uh
23:35
use those in their applications uh last very quick demo for the two minutes
23:37
last very quick demo for the two minutes
23:37
last very quick demo for the two minutes that i have left uh
23:38
that i have left uh
23:38
that i have left uh is a automation api
23:41
is a automation api
23:42
is a automation api the the demo i showed you before was
23:45
the the demo i showed you before was
23:45
the the demo i showed you before was using
23:45
using
23:45
using our serialized pluma cell you have to go
23:47
our serialized pluma cell you have to go
23:47
our serialized pluma cell you have to go to powershell or your other cli and
23:50
to powershell or your other cli and
23:50
to powershell or your other cli and run our command polyme
23:53
run our command polyme
23:53
run our command polyme what you can do instead is to drive the
23:55
what you can do instead is to drive the
23:55
what you can do instead is to drive the complete deployment
23:57
complete deployment
23:57
complete deployment from your chart program like in this
23:59
from your chart program like in this
23:59
from your chart program like in this case so i
24:01
case so i
24:01
case so i this speaks to the this code speaks to
24:04
this speaks to the this code speaks to
24:04
this speaks to the this code speaks to the
24:05
the
24:05
the case that i just described where you
24:06
case that i just described where you
24:06
case that i just described where you have a platform team and two
24:07
have a platform team and two
24:08
have a platform team and two applications team that build on top of
24:09
applications team that build on top of
24:09
applications team that build on top of it
24:10
it
24:10
it so it has three stacks one is a platform
24:13
so it has three stacks one is a platform
24:13
so it has three stacks one is a platform stack
24:13
stack
24:14
stack and two application stacks and this
24:17
and two application stacks and this
24:17
and two application stacks and this program defines just a collection of
24:18
program defines just a collection of
24:18
program defines just a collection of steps of deployment i want to deploy
24:21
steps of deployment i want to deploy
24:21
steps of deployment i want to deploy these three steps in sequence and then
24:24
these three steps in sequence and then
24:24
these three steps in sequence and then there is just a forage loop
24:26
there is just a forage loop
24:26
there is just a forage loop which says if it's not a destroy
24:27
which says if it's not a destroy
24:27
which says if it's not a destroy operation
24:29
operation
24:29
operation just go ahead and for each every step in
24:31
just go ahead and for each every step in
24:32
just go ahead and for each every step in this
24:33
this
24:33
this array just go ahead and up this command
24:36
array just go ahead and up this command
24:36
array just go ahead and up this command so
24:37
so
24:37
so the way you drive this is you just go to
24:39
the way you drive this is you just go to
24:39
the way you drive this is you just go to this command line
24:40
this command line
24:40
this command line you type dotnet run
24:44
you type dotnet run
24:44
you type dotnet run you execute this program there is no cli
24:46
you execute this program there is no cli
24:46
you execute this program there is no cli anywhere you can do it from your ci cd
24:48
anywhere you can do it from your ci cd
24:48
anywhere you can do it from your ci cd system you can write the web service
24:50
system you can write the web service
24:50
system you can write the web service which will do this
24:51
which will do this
24:51
which will do this on request like producing a new database
24:53
on request like producing a new database
24:53
on request like producing a new database per new customer of your product
24:56
per new customer of your product
24:56
per new customer of your product you can be as creative as you want and
24:58
you can be as creative as you want and
24:58
you can be as creative as you want and just drive
24:59
just drive
24:59
just drive all the whole deployment from your id
25:03
all the whole deployment from your id
25:03
all the whole deployment from your id but still be in full control and use the
25:07
but still be in full control and use the
25:07
but still be in full control and use the all the benefits of the desired state
25:09
all the benefits of the desired state
25:09
all the benefits of the desired state configuration just to quickly show you
25:11
configuration just to quickly show you
25:11
configuration just to quickly show you how this execution looks like i don't
25:13
how this execution looks like i don't
25:13
how this execution looks like i don't have time to actually run it maybe i can
25:15
have time to actually run it maybe i can
25:15
have time to actually run it maybe i can run it but don't wait until it finishes
25:17
run it but don't wait until it finishes
25:17
run it but don't wait until it finishes so i just type
25:20
so i just type
25:20
so i just type make it bigger again i'll just type
25:22
make it bigger again i'll just type
25:22
make it bigger again i'll just type dotnet run
25:23
dotnet run
25:23
dotnet run in this folder where i'm in the
25:25
in this folder where i'm in the
25:25
in this folder where i'm in the automation api folder
25:27
automation api folder
25:27
automation api folder and it runs the dotnet application that
25:29
and it runs the dotnet application that
25:29
and it runs the dotnet application that i have
25:30
i have
25:30
i have and it starts deploying my platform
25:32
and it starts deploying my platform
25:32
and it starts deploying my platform stack first
25:34
stack first
25:34
stack first uh then it will go ahead and deploy
25:36
uh then it will go ahead and deploy
25:36
uh then it will go ahead and deploy applications after this is done
25:37
applications after this is done
25:37
applications after this is done and you can control the output this is
25:39
and you can control the output this is
25:39
and you can control the output this is just some default output you can do like
25:42
just some default output you can do like
25:42
just some default output you can do like a
25:43
a
25:43
a windows application that that shows some
25:45
windows application that that shows some
25:45
windows application that that shows some progress if you want
25:47
progress if you want
25:47
progress if you want and so on and so forth it's just your
25:49
and so on and so forth it's just your
25:49
and so on and so forth it's just your imagination which is
25:50
imagination which is
25:50
imagination which is the limit here and your requirements
25:53
the limit here and your requirements
25:53
the limit here and your requirements so with that uh thank you
25:56
so with that uh thank you
25:56
so with that uh thank you very much for listening uh polymer is
25:59
very much for listening uh polymer is
25:59
very much for listening uh polymer is open source and free to use you can
26:00
open source and free to use you can
26:00
open source and free to use you can learn more
26:01
learn more
26:01
learn more about filmmakerlundy.com
26:04
about filmmakerlundy.com
26:04
about filmmakerlundy.com and i'm happy to answer any questions if
26:06
and i'm happy to answer any questions if
26:06
and i'm happy to answer any questions if you're here
26:08
you're here
26:08
you're here thank you
26:11
thank you
26:11
thank you wow awesome thank you very much wow um
26:15
wow awesome thank you very much wow um
26:15
wow awesome thank you very much wow um so i think we had a couple of questions
26:16
so i think we had a couple of questions
26:16
so i think we had a couple of questions but i'm going to be greedy and just ask
26:18
but i'm going to be greedy and just ask
26:18
but i'm going to be greedy and just ask my own question
26:19
my own question
26:19
my own question because this was all very interesting to
26:21
because this was all very interesting to
26:21
because this was all very interesting to me
26:22
me
26:22
me so it's a three-part question um so if
26:25
so it's a three-part question um so if
26:25
so it's a three-part question um so if you imagine
26:25
you imagine
26:25
you imagine um i'm new to infrared code and i'm
26:28
um i'm new to infrared code and i'm
26:28
um i'm new to infrared code and i'm beautiful about native databases
26:29
beautiful about native databases
26:30
beautiful about native databases or services um and i'm looking at this
26:32
or services um and i'm looking at this
26:32
or services um and i'm looking at this and my head's
26:33
and my head's
26:33
and my head's kind of exploding right now so i'm
26:35
kind of exploding right now so i'm
26:35
kind of exploding right now so i'm thinking i'm used to older tools like
26:37
thinking i'm used to older tools like
26:37
thinking i'm used to older tools like ansible maybe or maybe have manually
26:39
ansible maybe or maybe have manually
26:39
ansible maybe or maybe have manually provisioned
26:40
provisioned
26:40
provisioned environments or whatever what would be
26:42
environments or whatever what would be
26:42
environments or whatever what would be the number one thing
26:43
the number one thing
26:43
the number one thing number one reason that you would give
26:44
number one reason that you would give
26:44
number one reason that you would give them say you need to make this uh
26:46
them say you need to make this uh
26:46
them say you need to make this uh paradigm shift to doing this that's part
26:48
paradigm shift to doing this that's part
26:48
paradigm shift to doing this that's part one
26:48
one
26:48
one part two would be um is it any easier
26:51
part two would be um is it any easier
26:52
part two would be um is it any easier or why what makes it easier to use
26:53
or why what makes it easier to use
26:53
or why what makes it easier to use services like cosmos db with these
26:55
services like cosmos db with these
26:55
services like cosmos db with these approaches if it is easier
26:57
approaches if it is easier
26:57
approaches if it is easier and then the last thing would be any any
26:58
and then the last thing would be any any
26:58
and then the last thing would be any any pitfalls that you would say watch out
27:00
pitfalls that you would say watch out
27:00
pitfalls that you would say watch out for when you're using this approach
27:02
for when you're using this approach
27:02
for when you're using this approach that you might not expect from from what
27:04
that you might not expect from from what
27:04
that you might not expect from from what you've been used to in the past
27:07
you've been used to in the past
27:07
you've been used to in the past well to the first part you need to
27:09
well to the first part you need to
27:09
well to the first part you need to separate learning the
27:11
separate learning the
27:11
separate learning the infrastructure itself like learning what
27:14
infrastructure itself like learning what
27:14
infrastructure itself like learning what kubernetes is and what kind of uh
27:16
kubernetes is and what kind of uh
27:16
kubernetes is and what kind of uh objects you need to deploy there and
27:18
objects you need to deploy there and
27:18
objects you need to deploy there and like all the energy services that you
27:20
like all the energy services that you
27:20
like all the energy services that you need
27:21
need
27:21
need uh from learning the tool and i my point
27:24
uh from learning the tool and i my point
27:24
uh from learning the tool and i my point is that
27:25
is that
27:26
is that you sort of cannot avoid learning the
27:28
you sort of cannot avoid learning the
27:28
you sort of cannot avoid learning the infrastructure in your company somebody
27:29
infrastructure in your company somebody
27:30
infrastructure in your company somebody has to
27:30
has to
27:30
has to have this knowledge but then you don't
27:33
have this knowledge but then you don't
27:33
have this knowledge but then you don't have to
27:34
have to
27:34
have to make everyone learn that you can also
27:36
make everyone learn that you can also
27:36
make everyone learn that you can also just as i described have
27:38
just as i described have
27:38
just as i described have some people specialized on nitty-gritty
27:40
some people specialized on nitty-gritty
27:40
some people specialized on nitty-gritty details of those services and then
27:42
details of those services and then
27:42
details of those services and then they can explain to others and provide
27:44
they can explain to others and provide
27:44
they can explain to others and provide abstractions to others
27:46
abstractions to others
27:46
abstractions to others uh so that they wouldn't have to be
27:50
uh so that they wouldn't have to be
27:50
uh so that they wouldn't have to be kubernetes experts in order to be able
27:52
kubernetes experts in order to be able
27:52
kubernetes experts in order to be able to deploy their hello world
27:54
to deploy their hello world
27:54
to deploy their hello world with code and because application
27:56
with code and because application
27:56
with code and because application developers will be very well
27:59
developers will be very well
27:59
developers will be very well feel very familiar with the tools and
28:01
feel very familiar with the tools and
28:01
feel very familiar with the tools and like the the code
28:02
like the the code
28:02
like the the code and the c-sharp and so on they
28:05
and the c-sharp and so on they
28:05
and the c-sharp and so on they will be more willing to actually jump
28:08
will be more willing to actually jump
28:08
will be more willing to actually jump ahead and like
28:10
ahead and like
28:10
ahead and like start using this in my opinion
28:13
start using this in my opinion
28:13
start using this in my opinion so the cosmos db is is great i i try to
28:16
so the cosmos db is is great i i try to
28:16
so the cosmos db is is great i i try to illustrate this
28:18
illustrate this
28:18
illustrate this by using mongodb banggood api
28:20
by using mongodb banggood api
28:20
by using mongodb banggood api specifically
28:21
specifically
28:21
specifically because uh like you get all all the
28:25
because uh like you get all all the
28:25
because uh like you get all all the upsides of like the whole ecosystem
28:29
upsides of like the whole ecosystem
28:29
upsides of like the whole ecosystem built around mongodb when somebody built
28:30
built around mongodb when somebody built
28:30
built around mongodb when somebody built that home chart like they didn't know
28:32
that home chart like they didn't know
28:32
that home chart like they didn't know i'm going to use it with with
28:33
i'm going to use it with with
28:33
i'm going to use it with with cosmos db five years down the road
28:37
cosmos db five years down the road
28:37
cosmos db five years down the road but you still get all the benefits of uh
28:41
but you still get all the benefits of uh
28:41
but you still get all the benefits of uh very flexible cloud environments where
28:43
very flexible cloud environments where
28:43
very flexible cloud environments where you have operating in so you you can
28:45
you have operating in so you you can
28:45
you have operating in so you you can create cosmos db
28:47
create cosmos db
28:47
create cosmos db for your let's say you open a new pr for
28:49
for your let's say you open a new pr for
28:49
for your let's say you open a new pr for your system you will create a new
28:50
your system you will create a new
28:50
your system you will create a new environment with this tool
28:52
environment with this tool
28:52
environment with this tool like you run your tests you tear it down
28:55
like you run your tests you tear it down
28:55
like you run your tests you tear it down you pay maybe five cent i don't know
28:58
you pay maybe five cent i don't know
28:58
you pay maybe five cent i don't know like how much but you
28:59
like how much but you
28:59
like how much but you don't pay anything basically and uh
29:02
don't pay anything basically and uh
29:02
don't pay anything basically and uh you still got the mongodb which can be
29:04
you still got the mongodb which can be
29:04
you still got the mongodb which can be comparable to your production
29:05
comparable to your production
29:05
comparable to your production environment if you want to
29:07
environment if you want to
29:07
environment if you want to so you get like the full power of cloud
29:10
so you get like the full power of cloud
29:10
so you get like the full power of cloud and the
29:11
and the
29:11
and the power of third-party tooling or like
29:13
power of third-party tooling or like
29:13
power of third-party tooling or like community team that you
29:15
community team that you
29:15
community team that you get and uh the third question was the
29:18
get and uh the third question was the
29:18
get and uh the third question was the pitfalls i guess
29:19
pitfalls i guess
29:19
pitfalls i guess yeah the uh i guess the first part is
29:23
yeah the uh i guess the first part is
29:23
yeah the uh i guess the first part is like
29:23
like
29:23
like the main pitfall is that you need to get
29:26
the main pitfall is that you need to get
29:26
the main pitfall is that you need to get through
29:26
through
29:26
through the initial barrier of like being not
29:29
the initial barrier of like being not
29:29
the initial barrier of like being not too overwhelmed
29:30
too overwhelmed
29:30
too overwhelmed with all the uh all the
29:33
with all the uh all the
29:33
with all the uh all the offensive words that i've thrown in my
29:35
offensive words that i've thrown in my
29:35
offensive words that i've thrown in my presentation and like this is like
29:37
presentation and like this is like
29:37
presentation and like this is like one percent of all the things that exist
29:40
one percent of all the things that exist
29:40
one percent of all the things that exist out there so
29:42
out there so
29:42
out there so uh i i think like putting
29:45
uh i i think like putting
29:45
uh i i think like putting folks into a more familiar environment
29:47
folks into a more familiar environment
29:47
folks into a more familiar environment is a helpful
29:48
is a helpful
29:48
is a helpful but of course that's a learning curve
29:50
but of course that's a learning curve
29:50
but of course that's a learning curve that everybody has to take from like
29:52
that everybody has to take from like
29:52
that everybody has to take from like they shouldn't take it too lightly
29:58
cool one question that we got
30:01
cool one question that we got
30:01
cool one question that we got that's probably just worth mentioning
30:03
that's probably just worth mentioning
30:03
that's probably just worth mentioning for everyone viewing is
30:04
for everyone viewing is
30:04
for everyone viewing is what is the the best way to get started
30:08
what is the the best way to get started
30:08
what is the the best way to get started if you want you know if you're
30:09
if you want you know if you're
30:09
if you want you know if you're completely new to the
30:11
completely new to the
30:11
completely new to the infrastructure's code and you just want
30:13
infrastructure's code and you just want
30:13
infrastructure's code and you just want to get started from scratch
30:15
to get started from scratch
30:16
to get started from scratch so on my last slide there was a link
30:19
so on my last slide there was a link
30:19
so on my last slide there was a link polymer.com from there you can get
30:21
polymer.com from there you can get
30:22
polymer.com from there you can get to like a getting started guide for from
30:24
to like a getting started guide for from
30:24
to like a getting started guide for from there you just deploy
30:26
there you just deploy
30:26
there you just deploy a very simple like not that involved
30:29
a very simple like not that involved
30:29
a very simple like not that involved application that actually a very simple
30:31
application that actually a very simple
30:31
application that actually a very simple storage account with a static website or
30:33
storage account with a static website or
30:33
storage account with a static website or something and from there you can get
30:35
something and from there you can get
30:35
something and from there you can get started after
30:36
started after
30:36
started after you complete that tutorial and you
30:38
you complete that tutorial and you
30:38
you complete that tutorial and you already feel the workflow
30:40
already feel the workflow
30:40
already feel the workflow of how you provision new environment how
30:42
of how you provision new environment how
30:42
of how you provision new environment how you make
30:43
you make
30:43
you make changes how destroyed since you can
30:45
changes how destroyed since you can
30:45
changes how destroyed since you can already
30:46
already
30:46
already jump ahead to like more advanced
30:47
jump ahead to like more advanced
30:48
jump ahead to like more advanced scenarios based on your interests
30:49
scenarios based on your interests
30:49
scenarios based on your interests like cosmos db asia functions aks
30:53
like cosmos db asia functions aks
30:53
like cosmos db asia functions aks whatever um see what time you're
30:56
whatever um see what time you're
30:56
whatever um see what time you're uh thank you so much for your session
31:00
uh thank you so much for your session
31:00
uh thank you so much for your session um and thank you my pleasure yeah thanks
31:03
um and thank you my pleasure yeah thanks
31:03
um and thank you my pleasure yeah thanks mikkel
31:05
mikkel
31:05
mikkel next up um i'd like to introduce louis
31:08
next up um i'd like to introduce louis
31:08
next up um i'd like to introduce louis beltran and he's going to be talking
31:11
beltran and he's going to be talking
31:11
beltran and he's going to be talking about
31:12
about
31:12
about how to essentially set yourself free
31:14
how to essentially set yourself free
31:14
how to essentially set yourself free from your servers
31:15
from your servers
31:15
from your servers and what it takes to set up a serverless
31:18
and what it takes to set up a serverless
31:18
and what it takes to set up a serverless environment
31:18
environment
31:18
environment capable of performing crowd operations
31:21
capable of performing crowd operations
31:21
capable of performing crowd operations on a cosmos db account
31:22
on a cosmos db account
31:22
on a cosmos db account it's really exciting to hear this talk
31:23
it's really exciting to hear this talk
31:23
it's really exciting to hear this talk louis
31:25
louis
31:25
louis hello guys and good morning good
31:27
hello guys and good morning good
31:28
hello guys and good morning good afternoon everyone
31:29
afternoon everyone
31:29
afternoon everyone depending uh where you are joining or
31:31
depending uh where you are joining or
31:31
depending uh where you are joining or not i hope that you are enjoying the
31:33
not i hope that you are enjoying the
31:33
not i hope that you are enjoying the azure cosmos db con so let's get
31:37
azure cosmos db con so let's get
31:37
azure cosmos db con so let's get started
31:41
so my name is luis i am a microsoft
31:44
so my name is luis i am a microsoft
31:44
so my name is luis i am a microsoft mvp in ai and developer technologies i
31:47
mvp in ai and developer technologies i
31:47
mvp in ai and developer technologies i am based in czech republic i am
31:49
am based in czech republic i am
31:49
am based in czech republic i am currently
31:50
currently
31:50
currently pursuing my phd at thomas bata
31:52
pursuing my phd at thomas bata
31:52
pursuing my phd at thomas bata university in aslin
31:54
university in aslin
31:54
university in aslin i am also lecturer at technological
31:56
i am also lecturer at technological
31:56
i am also lecturer at technological national de mexico
31:57
national de mexico
31:57
national de mexico in celaya and along with some friends
32:00
in celaya and along with some friends
32:00
in celaya and along with some friends we'll lead the community
32:03
we'll lead the community
32:04
we'll lead the community summary community in spanish you can
32:06
summary community in spanish you can
32:06
summary community in spanish you can find us on facebook and also
32:08
find us on facebook and also
32:08
find us on facebook and also apprehending the usher learning azure
32:11
apprehending the usher learning azure
32:11
apprehending the usher learning azure here you have my contact details in case
32:13
here you have my contact details in case
32:13
here you have my contact details in case you would be interested in talking about
32:15
you would be interested in talking about
32:16
you would be interested in talking about these topics
32:17
these topics
32:17
these topics so let's get started
32:20
so let's get started
32:20
so let's get started you have been learning a lot about azure
32:23
you have been learning a lot about azure
32:23
you have been learning a lot about azure function
32:24
function
32:24
function about cosmos db so i will go
32:27
about cosmos db so i will go
32:27
about cosmos db so i will go directly to azure functions
32:30
directly to azure functions
32:30
directly to azure functions uh you know this is the classic model or
32:33
uh you know this is the classic model or
32:33
uh you know this is the classic model or hosting models that we have learned
32:36
hosting models that we have learned
32:36
hosting models that we have learned about the cloud there are
32:39
about the cloud there are
32:39
about the cloud there are if you have your on-premise environment
32:43
if you have your on-premise environment
32:43
if you have your on-premise environment as long as you are going into
32:46
as long as you are going into
32:46
as long as you are going into infrastructure as a service platform as
32:48
infrastructure as a service platform as
32:48
infrastructure as a service platform as a service
32:49
a service
32:49
a service or software as a service you are freeing
32:52
or software as a service you are freeing
32:52
or software as a service you are freeing up yourself with the responsibilities
32:57
up yourself with the responsibilities
32:57
up yourself with the responsibilities so in software as a service for instance
32:59
so in software as a service for instance
32:59
so in software as a service for instance you don't have any responsibility you
33:01
you don't have any responsibility you
33:01
you don't have any responsibility you just need maybe use a browser
33:03
just need maybe use a browser
33:03
just need maybe use a browser to start using an application such as
33:07
to start using an application such as
33:07
to start using an application such as office 365 documents and so on
33:10
office 365 documents and so on
33:10
office 365 documents and so on and if you provision some of these
33:13
and if you provision some of these
33:13
and if you provision some of these resources for instance a virtual machine
33:15
resources for instance a virtual machine
33:15
resources for instance a virtual machine well you are in infrastructure as a
33:17
well you are in infrastructure as a
33:17
well you are in infrastructure as a service
33:17
service
33:17
service so you don't worry about the
33:19
so you don't worry about the
33:19
so you don't worry about the virtualization technologies you won't
33:21
virtualization technologies you won't
33:21
virtualization technologies you won't worry about
33:21
worry about
33:22
worry about service videos go for the arrest you
33:26
service videos go for the arrest you
33:26
service videos go for the arrest you set up the operative system the runtime
33:28
set up the operative system the runtime
33:28
set up the operative system the runtime the data your applications and so on
33:32
the data your applications and so on
33:32
the data your applications and so on so there is a another model which is
33:35
so there is a another model which is
33:35
so there is a another model which is serverless and serverless basically
33:39
serverless and serverless basically
33:39
serverless and serverless basically means that you don't you want to write
33:42
means that you don't you want to write
33:42
means that you don't you want to write code
33:43
code
33:43
code you want to write applications but you
33:45
you want to write applications but you
33:45
you want to write applications but you don't want to worry about provisioning
33:48
don't want to worry about provisioning
33:48
don't want to worry about provisioning a virtual machine or resources
33:51
a virtual machine or resources
33:51
a virtual machine or resources so the cloud azure will do that for you
33:54
so the cloud azure will do that for you
33:54
so the cloud azure will do that for you so in this case in this scenario
33:57
so in this case in this scenario
33:57
so in this case in this scenario serverless
33:58
serverless
33:58
serverless yes you just worry you just manage your
34:00
yes you just worry you just manage your
34:00
yes you just worry you just manage your functions your codes
34:03
functions your codes
34:03
functions your codes the rest is already
34:06
the rest is already
34:06
the rest is already set up for you and you can use your
34:08
set up for you and you can use your
34:08
set up for you and you can use your function or you can integrate them
34:10
function or you can integrate them
34:10
function or you can integrate them in the other um
34:14
in the other um
34:14
in the other um uh in the other hosting models even even
34:17
uh in the other hosting models even even
34:17
uh in the other hosting models even even on premises and of course infrastructure
34:19
on premises and of course infrastructure
34:19
on premises and of course infrastructure as a service and
34:20
as a service and
34:20
as a service and platform as a service so
34:23
platform as a service so
34:24
platform as a service so azure functions basically works with
34:27
azure functions basically works with
34:27
azure functions basically works with events you you will write some code
34:30
events you you will write some code
34:30
events you you will write some code and this code will trigger
34:33
and this code will trigger
34:33
and this code will trigger on um the when some events happen
34:37
on um the when some events happen
34:37
on um the when some events happen so maybe you send an http request
34:41
so maybe you send an http request
34:41
so maybe you send an http request maybe you want to execute some codes um
34:43
maybe you want to execute some codes um
34:44
maybe you want to execute some codes um according to some
34:44
according to some
34:44
according to some schedule every hour every 10 hours and
34:47
schedule every hour every 10 hours and
34:47
schedule every hour every 10 hours and so
34:47
so
34:48
so on maybe when you
34:51
on maybe when you
34:51
on maybe when you when your users upload a picture into
34:54
when your users upload a picture into
34:54
when your users upload a picture into global storage you want to execute some
34:57
global storage you want to execute some
34:57
global storage you want to execute some calls
34:57
calls
34:57
calls for analysis or something else
35:01
for analysis or something else
35:01
for analysis or something else so okay you set up an event or you
35:05
so okay you set up an event or you
35:05
so okay you set up an event or you you know about an event then you write
35:08
you know about an event then you write
35:08
you know about an event then you write your code
35:09
your code
35:09
your code you have different options you can write
35:11
you have different options you can write
35:11
you have different options you can write your
35:12
your
35:12
your code in c sharp in
35:15
code in c sharp in
35:15
code in c sharp in f sharp and other languages
35:20
f sharp and other languages
35:20
f sharp and other languages and what's uh one of the
35:24
and what's uh one of the
35:24
and what's uh one of the good advantages is that you can bind
35:28
good advantages is that you can bind
35:28
good advantages is that you can bind some input and output of elements so you
35:31
some input and output of elements so you
35:31
some input and output of elements so you can directly connect for instance to
35:33
can directly connect for instance to
35:33
can directly connect for instance to cosmos tv which is the main topic of
35:36
cosmos tv which is the main topic of
35:36
cosmos tv which is the main topic of today's session
35:37
today's session
35:37
today's session uh you can also connect to blog storage
35:40
uh you can also connect to blog storage
35:40
uh you can also connect to blog storage service
35:41
service
35:41
service table storage or even you can send email
35:46
table storage or even you can send email
35:46
table storage or even you can send email to to after some processing
35:51
so your code plus
35:54
so your code plus
35:54
so your code plus events is actual functions basically
35:59
events is actual functions basically
35:59
events is actual functions basically well as we have mentioned uh yes there
36:02
well as we have mentioned uh yes there
36:02
well as we have mentioned uh yes there is full integration with azure services
36:04
is full integration with azure services
36:04
is full integration with azure services so um basically even grid we
36:08
so um basically even grid we
36:08
so um basically even grid we here will listen uh to some to will
36:12
here will listen uh to some to will
36:12
here will listen uh to some to will react to some events that are happening
36:14
react to some events that are happening
36:14
react to some events that are happening so for instance in databases in cosmos
36:16
so for instance in databases in cosmos
36:16
so for instance in databases in cosmos db
36:17
db
36:17
db uh when one of the items the documents
36:20
uh when one of the items the documents
36:20
uh when one of the items the documents are modified well you can also
36:22
are modified well you can also
36:22
are modified well you can also catch that event uh you can also
36:25
catch that event uh you can also
36:25
catch that event uh you can also integrate
36:26
integrate
36:26
integrate with other products such as cognitive
36:29
with other products such as cognitive
36:29
with other products such as cognitive services
36:31
services
36:31
services uh stream analytics it is actually
36:34
uh stream analytics it is actually
36:34
uh stream analytics it is actually a good idea to to use azure functions
36:37
a good idea to to use azure functions
36:38
a good idea to to use azure functions uh and integrated with iot solutions
36:42
uh and integrated with iot solutions
36:42
uh and integrated with iot solutions and also well if you don't want to write
36:45
and also well if you don't want to write
36:45
and also well if you don't want to write a
36:45
a
36:45
a code there is also a logic apps
36:49
code there is also a logic apps
36:49
code there is also a logic apps which allows you to design workflows
36:52
which allows you to design workflows
36:52
which allows you to design workflows what process will follow after
36:56
what process will follow after
36:56
what process will follow after some let's say events or after
37:00
some let's say events or after
37:00
some let's say events or after some interactions
37:04
and well yes there are several triggers
37:07
and well yes there are several triggers
37:07
and well yes there are several triggers i mentioned
37:08
i mentioned
37:08
i mentioned a few of them so http trigger is when
37:10
a few of them so http trigger is when
37:10
a few of them so http trigger is when you send
37:12
you send
37:12
you send a request over http like okay
37:16
a request over http like okay
37:16
a request over http like okay post put or even get
37:19
post put or even get
37:19
post put or even get there is also a timer so
37:22
there is also a timer so
37:22
there is also a timer so you set up a chrome um
37:26
you set up a chrome um
37:26
you set up a chrome um let's say a current statement and then
37:28
let's say a current statement and then
37:28
let's say a current statement and then it will execute every hour
37:30
it will execute every hour
37:30
it will execute every hour every monday at 7 00 am
37:34
every monday at 7 00 am
37:34
every monday at 7 00 am every day fifth of every month yeah
37:37
every day fifth of every month yeah
37:37
every day fifth of every month yeah you you can set up that uh so you
37:40
you you can set up that uh so you
37:40
you you can set up that uh so you run your code when the specified events
37:43
run your code when the specified events
37:43
run your code when the specified events happen
37:44
happen
37:44
happen so this is the example maybe you have a
37:47
so this is the example maybe you have a
37:47
so this is the example maybe you have a mobile application
37:48
mobile application
37:48
mobile application uh you take a photo the users take a
37:50
uh you take a photo the users take a
37:50
uh you take a photo the users take a photo they send it to the client
37:53
photo they send it to the client
37:53
photo they send it to the client and it is stored in love storage for
37:56
and it is stored in love storage for
37:56
and it is stored in love storage for instance
37:57
instance
37:57
instance and maybe you would like to create
38:00
and maybe you would like to create
38:00
and maybe you would like to create different versions of this
38:02
different versions of this
38:02
different versions of this picture maybe you want to create
38:04
picture maybe you want to create
38:04
picture maybe you want to create escalated versions uh
38:05
escalated versions uh
38:06
escalated versions uh resize them so in this case
38:09
resize them so in this case
38:09
resize them so in this case you can automate this task
38:13
you can automate this task
38:13
you can automate this task by setting up an azure function
38:16
by setting up an azure function
38:16
by setting up an azure function with an input uh let's say
38:20
with an input uh let's say
38:20
with an input uh let's say element which is the blob storage and
38:23
element which is the blob storage and
38:23
element which is the blob storage and also an
38:23
also an
38:23
also an output because you want to create some
38:25
output because you want to create some
38:25
output because you want to create some files and store them maybe
38:27
files and store them maybe
38:27
files and store them maybe in a different container okay so
38:31
in a different container okay so
38:31
in a different container okay so and actually if you are wondering okay
38:33
and actually if you are wondering okay
38:33
and actually if you are wondering okay is this code
38:34
is this code
38:34
is this code uh hard is this called complicated well
38:38
uh hard is this called complicated well
38:38
uh hard is this called complicated well the truth is that no and this is
38:41
the truth is that no and this is
38:41
the truth is that no and this is because of the bindings because well you
38:44
because of the bindings because well you
38:44
because of the bindings because well you have your code for instance this is your
38:45
have your code for instance this is your
38:45
have your code for instance this is your c-sharp code
38:46
c-sharp code
38:46
c-sharp code and there is a csx file and you have
38:50
and there is a csx file and you have
38:50
and there is a csx file and you have your bindings your bindings and are said
38:53
your bindings your bindings and are said
38:53
your bindings your bindings and are said you have a designer you can do it
38:55
you have a designer you can do it
38:55
you have a designer you can do it graphically you can select and connect
38:57
graphically you can select and connect
38:57
graphically you can select and connect by using let's say a connection string
38:59
by using let's say a connection string
38:59
by using let's say a connection string or
39:00
or
39:00
or a parameter but this information is
39:02
a parameter but this information is
39:02
a parameter but this information is stored in a function.json
39:05
stored in a function.json
39:06
stored in a function.json file so you see
39:09
file so you see
39:09
file so you see that we have an image a binding
39:13
that we have an image a binding
39:13
that we have an image a binding it's an input you see the direction uh
39:16
it's an input you see the direction uh
39:16
it's an input you see the direction uh the second one is output direction so
39:18
the second one is output direction so
39:18
the second one is output direction so this one is output binding
39:20
this one is output binding
39:20
this one is output binding then you have the path of course the
39:22
then you have the path of course the
39:22
then you have the path of course the connection
39:23
connection
39:24
connection tells you okay i am using this global
39:27
tells you okay i am using this global
39:27
tells you okay i am using this global storage uh
39:29
storage uh
39:29
storage uh element uh or sorry storage account
39:32
element uh or sorry storage account
39:32
element uh or sorry storage account and in the storage account there is a
39:34
and in the storage account there is a
39:34
and in the storage account there is a container the container is card
39:36
container the container is card
39:36
container the container is card input and then the file name this is the
39:39
input and then the file name this is the
39:39
input and then the file name this is the file that
39:40
file that
39:40
file that the user just uploaded so
39:43
the user just uploaded so
39:43
the user just uploaded so these two elements such as the name and
39:46
these two elements such as the name and
39:46
these two elements such as the name and the
39:47
the
39:47
the path can be used as parameters or my of
39:50
path can be used as parameters or my of
39:50
path can be used as parameters or my of my function
39:51
my function
39:51
my function so the byte image of course
39:55
so the byte image of course
39:55
so the byte image of course means the element that just came in that
39:58
means the element that just came in that
39:58
means the element that just came in that was just inserted
40:00
was just inserted
40:00
was just inserted and uh the string filing of course
40:03
and uh the string filing of course
40:03
and uh the string filing of course refers to the
40:03
refers to the
40:04
refers to the um the name of the of this block
40:07
um the name of the of this block
40:07
um the name of the of this block then you have the same for the um
40:11
then you have the same for the um
40:11
then you have the same for the um output parameter maybe you i have output
40:13
output parameter maybe you i have output
40:13
output parameter maybe you i have output block okay that's the reference
40:15
block okay that's the reference
40:15
block okay that's the reference that's the stream where i want to save
40:17
that's the stream where i want to save
40:17
that's the stream where i want to save the
40:18
the
40:18
the the file and i want to use maybe the
40:21
the file and i want to use maybe the
40:21
the file and i want to use maybe the same
40:21
same
40:21
same file name but in assets as you can see
40:24
file name but in assets as you can see
40:24
file name but in assets as you can see in different container
40:25
in different container
40:25
in different container they both use the same connection and
40:28
they both use the same connection and
40:28
they both use the same connection and you can see the code is quite simple
40:30
you can see the code is quite simple
40:30
you can see the code is quite simple just a few lines of code you don't have
40:32
just a few lines of code you don't have
40:32
just a few lines of code you don't have to
40:33
to
40:33
to create like a client for connecting to
40:36
create like a client for connecting to
40:36
create like a client for connecting to your
40:37
your
40:37
your input blob storage and for the output
40:39
input blob storage and for the output
40:39
input blob storage and for the output blob storage
40:41
blob storage
40:41
blob storage it's quite easy thanks to the bindings
40:43
it's quite easy thanks to the bindings
40:43
it's quite easy thanks to the bindings and the code of course
40:47
what tools can you use to write actual
40:49
what tools can you use to write actual
40:49
what tools can you use to write actual functions
40:50
functions
40:50
functions well you can even do it in the portal as
40:52
well you can even do it in the portal as
40:52
well you can even do it in the portal as that's the case that i'm going to
40:55
that's the case that i'm going to
40:55
that's the case that i'm going to demonstrate today but you can also use
40:58
demonstrate today but you can also use
40:58
demonstrate today but you can also use visual studio ps code the cli and some
41:02
visual studio ps code the cli and some
41:02
visual studio ps code the cli and some other
41:02
other
41:02
other deployment options actually if you
41:04
deployment options actually if you
41:04
deployment options actually if you remember
41:05
remember
41:05
remember in previous slide we talked about the
41:08
in previous slide we talked about the
41:08
in previous slide we talked about the on-premises
41:09
on-premises
41:09
on-premises hosting model uh you can
41:12
hosting model uh you can
41:12
hosting model uh you can have your server-less functions
41:15
have your server-less functions
41:15
have your server-less functions um running in in on-premises environment
41:19
um running in in on-premises environment
41:19
um running in in on-premises environment so you don't even need to deploy them to
41:21
so you don't even need to deploy them to
41:21
so you don't even need to deploy them to the cloud they can
41:23
the cloud they can
41:23
the cloud they can work and you can even deploy them to to
41:25
work and you can even deploy them to to
41:25
work and you can even deploy them to to containers
41:28
so what happens with cosmos db and azure
41:31
so what happens with cosmos db and azure
41:31
so what happens with cosmos db and azure functions
41:33
functions
41:33
functions well with the negative integration
41:35
well with the negative integration
41:35
well with the negative integration between
41:36
between
41:36
between azure cosmos db and azure functions
41:39
azure cosmos db and azure functions
41:39
azure cosmos db and azure functions there are
41:39
there are
41:39
there are different ways to to to interact
41:42
different ways to to to interact
41:42
different ways to to to interact between them there are ways database
41:44
between them there are ways database
41:44
between them there are ways database triggers input bindings and output
41:46
triggers input bindings and output
41:46
triggers input bindings and output bindings
41:47
bindings
41:47
bindings directly from your azure cosmos tv
41:49
directly from your azure cosmos tv
41:49
directly from your azure cosmos tv account
41:50
account
41:50
account so this means that you can deploy you
41:53
so this means that you can deploy you
41:53
so this means that you can deploy you can create
41:54
can create
41:54
can create serverless application that
41:58
serverless application that
41:58
serverless application that are based or rely on events with low
42:01
are based or rely on events with low
42:01
are based or rely on events with low latency
42:01
latency
42:01
latency access to rich data for global use
42:05
access to rich data for global use
42:05
access to rich data for global use service
42:05
service
42:05
service yeah basically cosmos db
42:08
yeah basically cosmos db
42:08
yeah basically cosmos db so yeah there are three ways
42:12
so yeah there are three ways
42:12
so yeah there are three ways the first one is you can create an azure
42:15
the first one is you can create an azure
42:15
the first one is you can create an azure functions triggers
42:15
functions triggers
42:16
functions triggers which is event driven which relies on
42:18
which is event driven which relies on
42:18
which is event driven which relies on the change feed
42:19
the change feed
42:19
the change feed streams to monitor your container for
42:22
streams to monitor your container for
42:22
streams to monitor your container for changes
42:22
changes
42:22
changes so when there are some changes made
42:25
so when there are some changes made
42:25
so when there are some changes made to the container this change feed stream
42:29
to the container this change feed stream
42:29
to the container this change feed stream is sent to the trigger and thus
42:32
is sent to the trigger and thus
42:32
is sent to the trigger and thus invoking the azure function actually
42:36
invoking the azure function actually
42:36
invoking the azure function actually there is also a nice another nice
42:39
there is also a nice another nice
42:39
there is also a nice another nice talk in this cosmos db conf
42:42
talk in this cosmos db conf
42:42
talk in this cosmos db conf by gabriela martinez she talked about
42:45
by gabriela martinez she talked about
42:45
by gabriela martinez she talked about the change feed so i recommend you to
42:47
the change feed so i recommend you to
42:47
the change feed so i recommend you to watch that presentation the second one
42:51
watch that presentation the second one
42:51
watch that presentation the second one is prepare the input binding
42:55
is prepare the input binding
42:55
is prepare the input binding so you set up the connection for your
42:58
so you set up the connection for your
42:58
so you set up the connection for your azure cosmos db
43:00
azure cosmos db
43:00
azure cosmos db and you can use this element you will
43:04
and you can use this element you will
43:04
and you can use this element you will see
43:04
see
43:04
see with very few lines of codes and
43:07
with very few lines of codes and
43:07
with very few lines of codes and retrieve the data that you have in your
43:10
retrieve the data that you have in your
43:10
retrieve the data that you have in your cosmos db
43:12
cosmos db
43:12
cosmos db and yeah that's that would be the input
43:14
and yeah that's that would be the input
43:14
and yeah that's that would be the input binding it will read
43:16
binding it will read
43:16
binding it will read data from the container and when the
43:18
data from the container and when the
43:18
data from the container and when the function executes
43:20
function executes
43:20
function executes and the third case is bind a function
43:23
and the third case is bind a function
43:23
and the third case is bind a function to another custom cosmos container with
43:26
to another custom cosmos container with
43:26
to another custom cosmos container with that output binding
43:28
that output binding
43:28
that output binding you will use this when you want to write
43:30
you will use this when you want to write
43:30
you will use this when you want to write data for instance when you want to
43:32
data for instance when you want to
43:32
data for instance when you want to insert a new
43:33
insert a new
43:33
insert a new element okay so
43:36
element okay so
43:36
element okay so let's go for that demo and just give me
43:39
let's go for that demo and just give me
43:39
let's go for that demo and just give me a second
43:40
a second
43:40
a second please i will bring my
43:45
um this i will bring here
43:50
um this i will bring here
43:50
um this i will bring here my um i have already set up
43:53
my um i have already set up
43:54
my um i have already set up a cosmos db uh
43:57
a cosmos db uh
43:57
a cosmos db uh database so an account so you can see in
43:59
database so an account so you can see in
43:59
database so an account so you can see in this case i have
44:02
this case i have
44:02
this case i have three members yes i have this
44:05
three members yes i have this
44:05
three members yes i have this uh informatics in this one okay and yes
44:10
i have these three elements
44:15
so um i would like to
44:19
so um i would like to
44:19
so um i would like to uh check text okay give
44:26
yes so we have our
44:30
yes so we have our
44:30
yes so we have our resource group
44:33
and i have already set up some azure
44:35
and i have already set up some azure
44:35
and i have already set up some azure functions
44:38
which is here university functions
44:46
so if i go to my functions
44:52
i will have this for instance these get
44:55
i will have this for instance these get
44:55
i will have this for instance these get students all of them are http
45:00
students all of them are http
45:00
students all of them are http triggers and uh well i
45:04
triggers and uh well i
45:04
triggers and uh well i first want you to look at the
45:07
first want you to look at the
45:07
first want you to look at the integration before
45:08
integration before
45:08
integration before looking at the code so in this
45:10
looking at the code so in this
45:10
looking at the code so in this integration i can easily add
45:13
integration i can easily add
45:13
integration i can easily add input or output bindings in this case
45:16
input or output bindings in this case
45:16
input or output bindings in this case it's a get student so i want to read
45:18
it's a get student so i want to read
45:18
it's a get student so i want to read information from my
45:19
information from my
45:20
information from my database so you can add
45:23
database so you can add
45:23
database so you can add an input and when you add the input you
45:26
an input and when you add the input you
45:26
an input and when you add the input you pick
45:26
pick
45:26
pick from the list you have uh as you can see
45:29
from the list you have uh as you can see
45:29
from the list you have uh as you can see you you well i cannot modify it of
45:31
you you well i cannot modify it of
45:31
you you well i cannot modify it of course
45:32
course
45:32
course but you have a blob trigger you have
45:34
but you have a blob trigger you have
45:34
but you have a blob trigger you have azure cosmos db
45:35
azure cosmos db
45:35
azure cosmos db then you set up some document parameter
45:38
then you set up some document parameter
45:38
then you set up some document parameter name this you will use it in your code
45:40
name this you will use it in your code
45:40
name this you will use it in your code in this my case it's input document
45:42
in this my case it's input document
45:42
in this my case it's input document that's the reference
45:44
that's the reference
45:44
that's the reference for the parameter the database name is
45:47
for the parameter the database name is
45:47
for the parameter the database name is studentsdb my collection name is
45:49
studentsdb my collection name is
45:50
studentsdb my collection name is students
45:50
students
45:50
students and this is my connection uh that i
45:54
and this is my connection uh that i
45:54
and this is my connection uh that i created for
45:54
created for
45:54
created for my cosmos tv if i don't have it at the
45:57
my cosmos tv if i don't have it at the
45:57
my cosmos tv if i don't have it at the beginning of course i can create it
45:59
beginning of course i can create it
45:59
beginning of course i can create it and it will detect in this case i have
46:01
and it will detect in this case i have
46:01
and it will detect in this case i have the
46:02
the
46:02
the the cosmos db
46:05
the cosmos db
46:05
the cosmos db account connection sorry account in the
46:07
account connection sorry account in the
46:07
account connection sorry account in the same resource group
46:09
same resource group
46:10
same resource group and i can set up document id if i want
46:12
and i can set up document id if i want
46:12
and i can set up document id if i want to and my partition key which is
46:14
to and my partition key which is
46:14
to and my partition key which is optional but
46:14
optional but
46:14
optional but it's nice to serve it up from the
46:16
it's nice to serve it up from the
46:16
it's nice to serve it up from the beginning stop its faculty
46:18
beginning stop its faculty
46:18
beginning stop its faculty and that's it then look at the code and
46:22
and that's it then look at the code and
46:22
and that's it then look at the code and it's
46:22
it's
46:22
it's quite simple well first
46:26
quite simple well first
46:26
quite simple well first i just need to um
46:29
i just need to um
46:29
i just need to um well you don't need actually all these
46:31
well you don't need actually all these
46:31
well you don't need actually all these uh all the
46:33
uh all the
46:33
uh all the namespaces but i added them but you
46:35
namespaces but i added them but you
46:35
namespaces but i added them but you don't need them uh
46:36
don't need them uh
46:36
don't need them uh i just set my
46:40
i just set my
46:40
i just set my let's say um my class student which
46:44
let's say um my class student which
46:44
let's say um my class student which with the properties then
46:47
with the properties then
46:47
with the properties then you can see in the parameter i just have
46:50
you can see in the parameter i just have
46:50
you can see in the parameter i just have the input document
46:52
the input document
46:52
the input document which i mentioned earlier
46:55
which i mentioned earlier
46:56
which i mentioned earlier i set this one as a list or in variable
47:00
i set this one as a list or in variable
47:00
i set this one as a list or in variable of
47:00
of
47:00
of students then the code is quite simple
47:04
students then the code is quite simple
47:04
students then the code is quite simple convert maybe you're into document into
47:07
convert maybe you're into document into
47:07
convert maybe you're into document into a list of a student
47:08
a list of a student
47:08
a list of a student and you can return it that's it
47:12
and you can return it that's it
47:12
and you can return it that's it then you we can test it uh measure
47:14
then you we can test it uh measure
47:14
then you we can test it uh measure function provides a nice
47:16
function provides a nice
47:16
function provides a nice console to um to prepare a request to
47:20
console to um to prepare a request to
47:20
console to um to prepare a request to get post you can also use other tools
47:22
get post you can also use other tools
47:22
get post you can also use other tools such as postman if you want
47:24
such as postman if you want
47:24
such as postman if you want and that's it i don't need to bring any
47:26
and that's it i don't need to bring any
47:26
and that's it i don't need to bring any parameters
47:27
parameters
47:27
parameters i can leave it so i will run this
47:31
i can leave it so i will run this
47:31
i can leave it so i will run this and we will get the information
47:35
and we will get the information
47:35
and we will get the information so yeah so you see
47:38
so yeah so you see
47:38
so yeah so you see this is my data it's already in very
47:40
this is my data it's already in very
47:40
this is my data it's already in very nice json
47:41
nice json
47:41
nice json style so yeah i
47:44
style so yeah i
47:44
style so yeah i have brought all my information from my
47:47
have brought all my information from my
47:47
have brought all my information from my database
47:47
database
47:47
database that's it then you might wonder okay
47:50
that's it then you might wonder okay
47:50
that's it then you might wonder okay what
47:51
what
47:51
what okay so this is nice it's in the portal
47:53
okay so this is nice it's in the portal
47:53
okay so this is nice it's in the portal but how can i
47:55
but how can i
47:55
but how can i connect it to my application well for
47:58
connect it to my application well for
47:58
connect it to my application well for functions
47:59
functions
47:59
functions in this case since since it is an http
48:02
in this case since since it is an http
48:02
in this case since since it is an http trigger
48:03
trigger
48:03
trigger it means that it will only execute when
48:05
it means that it will only execute when
48:05
it means that it will only execute when i send a request
48:07
i send a request
48:07
i send a request so there is get function url
48:10
so there is get function url
48:10
so there is get function url and in this case i have this
48:13
and in this case i have this
48:14
and in this case i have this this url i can
48:17
this url i can
48:17
this url i can maybe integrate it in my application or
48:19
maybe integrate it in my application or
48:19
maybe integrate it in my application or i can
48:20
i can
48:20
i can check it in the browser and you will see
48:22
check it in the browser and you will see
48:22
check it in the browser and you will see the data
48:24
the data
48:24
the data so yeah that's just it it's the same so
48:27
so yeah that's just it it's the same so
48:27
so yeah that's just it it's the same so yeah now you know how to integrate it
48:30
yeah now you know how to integrate it
48:30
yeah now you know how to integrate it and what about
48:31
and what about
48:31
and what about some other operations well in this case
48:33
some other operations well in this case
48:34
some other operations well in this case i have also prepared
48:35
i have also prepared
48:35
i have also prepared as a student which is
48:39
as a student which is
48:39
as a student which is this one so
48:42
this one so
48:42
this one so it's more or less the same in this case
48:44
it's more or less the same in this case
48:44
it's more or less the same in this case i want to actually write information
48:47
i want to actually write information
48:47
i want to actually write information so uh the first thing is okay i create
48:50
so uh the first thing is okay i create
48:50
so uh the first thing is okay i create my
48:51
my
48:51
my function then i go to integration and in
48:54
function then i go to integration and in
48:54
function then i go to integration and in integration
48:54
integration
48:54
integration i don't want input trigger now i want
48:57
i don't want input trigger now i want
48:58
i don't want input trigger now i want output sorry not input binding output
49:00
output sorry not input binding output
49:00
output sorry not input binding output binding
49:01
binding
49:01
binding and you see that in this case the output
49:03
and you see that in this case the output
49:03
and you see that in this case the output i have already set up
49:04
i have already set up
49:04
i have already set up and i should cosmos db output document
49:08
and i should cosmos db output document
49:08
and i should cosmos db output document and let's just briefly explore it it's
49:10
and let's just briefly explore it it's
49:10
and let's just briefly explore it it's basically the same
49:12
basically the same
49:12
basically the same but now it's an output element so
49:15
but now it's an output element so
49:15
but now it's an output element so here we have um the binding type of
49:17
here we have um the binding type of
49:17
here we have um the binding type of course
49:18
course
49:18
course uh this is the reference that i will use
49:21
uh this is the reference that i will use
49:21
uh this is the reference that i will use as parameter in my calls
49:23
as parameter in my calls
49:23
as parameter in my calls and the rest you know it database name
49:25
and the rest you know it database name
49:25
and the rest you know it database name collection name
49:27
collection name
49:27
collection name i don't want to create the database it
49:29
i don't want to create the database it
49:29
i don't want to create the database it already exists
49:30
already exists
49:30
already exists and my connection and my question that's
49:33
and my connection and my question that's
49:33
and my connection and my question that's it
49:34
it
49:34
it then the code yeah it's also
49:38
then the code yeah it's also
49:38
then the code yeah it's also simple you will even see
49:43
so okay yeah um
49:46
so okay yeah um
49:46
so okay yeah um the same i have my my class
49:49
the same i have my my class
49:49
the same i have my my class and here i have the reference my output
49:53
and here i have the reference my output
49:53
and here i have the reference my output document
49:53
document
49:54
document it's an out parameter
49:58
it's an out parameter
49:58
it's an out parameter in this case i need to provide the
50:00
in this case i need to provide the
50:00
in this case i need to provide the information i need to
50:01
information i need to
50:02
information i need to send a post request with the
50:05
send a post request with the
50:05
send a post request with the information yes that i want to
50:08
information yes that i want to
50:08
information yes that i want to insert then i
50:11
insert then i
50:12
insert then i okay so i get the body i convert it yes
50:14
okay so i get the body i convert it yes
50:14
okay so i get the body i convert it yes i deserve less it
50:16
i deserve less it
50:16
i deserve less it and then i set the value for output
50:20
and then i set the value for output
50:20
and then i set the value for output document
50:21
document
50:21
document and new and then i provide every a bit
50:24
and new and then i provide every a bit
50:24
and new and then i provide every a bit of information that i
50:26
of information that i
50:26
of information that i will get from my post request and that's
50:28
will get from my post request and that's
50:28
will get from my post request and that's it
50:30
it
50:30
it as soon as you set up this and of course
50:33
as soon as you set up this and of course
50:33
as soon as you set up this and of course uh finish your code
50:37
uh finish your code
50:37
uh finish your code it will be written to the cosmos db
50:40
it will be written to the cosmos db
50:40
it will be written to the cosmos db so we can test it
50:44
so for this i have already this
50:48
so for this i have already this
50:48
so for this i have already this value okay i will change it let me
50:51
value okay i will change it let me
50:51
value okay i will change it let me put some other information so faculty
50:55
put some other information so faculty
50:55
put some other information so faculty is technology
50:59
and let's go for
51:03
i don't know some parts
51:10
so let's run it
51:14
okay
51:20
okay so the response is 200 yes well in
51:23
okay so the response is 200 yes well in
51:23
okay so the response is 200 yes well in this case i didn't set a block i just
51:25
this case i didn't set a block i just
51:25
this case i didn't set a block i just get a response but i can
51:28
get a response but i can
51:28
get a response but i can go back here to my get students
51:32
go back here to my get students
51:32
go back here to my get students and now i have four data i have new
51:35
and now i have four data i have new
51:35
and now i have four data i have new elements
51:36
elements
51:36
elements okay so yeah
51:40
okay so yeah
51:40
okay so yeah then for um update and delete
51:43
then for um update and delete
51:43
then for um update and delete well i have an issue there so i will not
51:47
well i have an issue there so i will not
51:47
well i have an issue there so i will not show this uh well it should work but for
51:50
show this uh well it should work but for
51:50
show this uh well it should work but for some reason i am getting some some error
51:51
some reason i am getting some some error
51:52
some reason i am getting some some error in deletes but i can share this later
51:54
in deletes but i can share this later
51:54
in deletes but i can share this later in my github um repository
51:58
in my github um repository
51:58
in my github um repository uh but uh yeah it should uh be also
52:01
uh but uh yeah it should uh be also
52:01
uh but uh yeah it should uh be also it's just some minor detail there but
52:04
it's just some minor detail there but
52:04
it's just some minor detail there but of course the idea here is that you
52:08
of course the idea here is that you
52:08
of course the idea here is that you create this very easily as you can see
52:10
create this very easily as you can see
52:10
create this very easily as you can see the code is
52:11
the code is
52:11
the code is quite short you just set up you can set
52:14
quite short you just set up you can set
52:14
quite short you just set up you can set up in minutes
52:15
up in minutes
52:15
up in minutes and well yeah also the same goes for
52:18
and well yeah also the same goes for
52:18
and well yeah also the same goes for this ad student there is also a function
52:21
this ad student there is also a function
52:21
this ad student there is also a function url
52:24
url
52:24
url which i have well okay i click
52:27
which i have well okay i click
52:27
which i have well okay i click but i shouldn't have um
52:30
but i shouldn't have um
52:30
but i shouldn't have um so yeah just one second i get different
52:33
so yeah just one second i get different
52:33
so yeah just one second i get different uh function url
52:37
uh function url
52:37
uh function url but of course i cannot test this one
52:39
but of course i cannot test this one
52:39
but of course i cannot test this one directly here because i need to
52:41
directly here because i need to
52:41
directly here because i need to bring some information in the body for
52:44
bring some information in the body for
52:44
bring some information in the body for instance okay so but
52:45
instance okay so but
52:45
instance okay so but yeah so that's why this one will not
52:48
yeah so that's why this one will not
52:48
yeah so that's why this one will not directly work here
52:51
so yeah essentially that will be all
52:54
so yeah essentially that will be all
52:54
so yeah essentially that will be all uh yes as i said well i will
52:57
uh yes as i said well i will
52:57
uh yes as i said well i will complete that part the update and they
52:59
complete that part the update and they
52:59
complete that part the update and they delete which is giving me some some
53:01
delete which is giving me some some
53:01
delete which is giving me some some issues but
53:02
issues but
53:02
issues but i will fix it so the call to action i
53:05
i will fix it so the call to action i
53:05
i will fix it so the call to action i share this
53:06
share this
53:06
share this couple of links you can take a
53:07
couple of links you can take a
53:07
couple of links you can take a screenshot or i can share the
53:09
screenshot or i can share the
53:09
screenshot or i can share the presentation later as well
53:11
presentation later as well
53:11
presentation later as well there is serverless database computing
53:12
there is serverless database computing
53:12
there is serverless database computing with azure cosmos db
53:14
with azure cosmos db
53:14
with azure cosmos db measure functions there is this link and
53:17
measure functions there is this link and
53:17
measure functions there is this link and also
53:17
also
53:17
also the video that i referenced a bit
53:20
the video that i referenced a bit
53:20
the video that i referenced a bit earlier by gabriella martinez for the
53:23
earlier by gabriella martinez for the
53:23
earlier by gabriella martinez for the azure cosmos db conf
53:24
azure cosmos db conf
53:24
azure cosmos db conf integrating asset cosmos tv with azure
53:26
integrating asset cosmos tv with azure
53:26
integrating asset cosmos tv with azure functions
53:28
functions
53:28
functions so that will be all from me thank you
53:31
so that will be all from me thank you
53:31
so that will be all from me thank you very much for
53:32
very much for
53:32
very much for having me please if you have any
53:34
having me please if you have any
53:34
having me please if you have any questions
53:35
questions
53:35
questions i will do my best to answer them here
53:38
i will do my best to answer them here
53:38
i will do my best to answer them here you also have my contact details so
53:40
you also have my contact details so
53:40
you also have my contact details so yeah i'm up for questions back to you
53:43
yeah i'm up for questions back to you
53:43
yeah i'm up for questions back to you guys thank you
53:45
guys thank you
53:45
guys thank you great thank you so much luis for a great
53:47
great thank you so much luis for a great
53:47
great thank you so much luis for a great presentation you made it really clear
53:49
presentation you made it really clear
53:49
presentation you made it really clear how the complementary benefits of using
53:52
how the complementary benefits of using
53:52
how the complementary benefits of using azure customers db
53:53
azure customers db
53:53
azure customers db and functions work together to really
53:56
and functions work together to really
53:56
and functions work together to really build powerful applications that was
53:57
build powerful applications that was
53:57
build powerful applications that was really cool
53:58
really cool
53:58
really cool um one question we got in the chat was
54:01
um one question we got in the chat was
54:01
um one question we got in the chat was around performance and what
54:04
around performance and what
54:04
around performance and what recommendations do you have for users
54:07
recommendations do you have for users
54:07
recommendations do you have for users who want to
54:07
who want to
54:08
who want to literally leverage functions and
54:11
literally leverage functions and
54:11
literally leverage functions and cosmos db in terms of performance yes
54:14
cosmos db in terms of performance yes
54:14
cosmos db in terms of performance yes that's a good question thank you
54:16
that's a good question thank you
54:16
that's a good question thank you well it is true that serverless
54:22
has some um requirements like for
54:24
has some um requirements like for
54:24
has some um requirements like for instance
54:25
instance
54:25
instance if your application does or your
54:28
if your application does or your
54:28
if your application does or your function doesn't uh get a request
54:32
function doesn't uh get a request
54:32
function doesn't uh get a request uh after 10 minutes it goes let's say
54:35
uh after 10 minutes it goes let's say
54:35
uh after 10 minutes it goes let's say back to sleep
54:37
back to sleep
54:37
back to sleep so next time that you bring it up with
54:40
so next time that you bring it up with
54:40
so next time that you bring it up with that you send another request
54:42
that you send another request
54:42
that you send another request it will take some time to to uh
54:45
it will take some time to to uh
54:45
it will take some time to to uh to wake up and to execute
54:48
to wake up and to execute
54:48
to wake up and to execute your code i have a small trick
54:51
your code i have a small trick
54:51
your code i have a small trick for that i uh always keep
54:55
for that i uh always keep
54:55
for that i uh always keep the let's say the server running or the
54:57
the let's say the server running or the
54:57
the let's say the server running or the virtual machine running where your the
54:59
virtual machine running where your the
54:59
virtual machine running where your the code is host
55:00
code is host
55:00
code is host um i also set a timer trigger
55:04
um i also set a timer trigger
55:04
um i also set a timer trigger that is executing every nine minutes
55:08
that is executing every nine minutes
55:08
that is executing every nine minutes so the my server is always running
55:11
so the my server is always running
55:11
so the my server is always running you might wonder okay but is this going
55:13
you might wonder okay but is this going
55:14
you might wonder okay but is this going to be a costly
55:15
to be a costly
55:15
to be a costly uh solution well the truth is that uh
55:18
uh solution well the truth is that uh
55:18
uh solution well the truth is that uh azure functions uh has a micro building
55:21
azure functions uh has a micro building
55:21
azure functions uh has a micro building uh let's say um
55:25
feature which means that you only get
55:28
feature which means that you only get
55:28
feature which means that you only get built
55:30
built
55:30
built uh by the time that your code is
55:34
uh by the time that your code is
55:34
uh by the time that your code is running i mean executed
55:38
running i mean executed
55:38
running i mean executed even if your code is always hosted there
55:40
even if your code is always hosted there
55:40
even if your code is always hosted there you don't get
55:41
you don't get
55:41
you don't get built as as it would be for a virtual
55:44
built as as it would be for a virtual
55:44
built as as it would be for a virtual machine
55:45
machine
55:45
machine or for an app service so
55:49
or for an app service so
55:49
or for an app service so and the cost is you you can get maybe
55:51
and the cost is you you can get maybe
55:51
and the cost is you you can get maybe even less than one dollar per month
55:54
even less than one dollar per month
55:54
even less than one dollar per month so i i think i have got got a good
55:58
so i i think i have got got a good
55:58
so i i think i have got got a good results i mean regarding performance you
56:01
results i mean regarding performance you
56:01
results i mean regarding performance you can also create
56:02
can also create
56:02
can also create azure functions instead of serverless
56:04
azure functions instead of serverless
56:04
azure functions instead of serverless you can set it as an app service if you
56:07
you can set it as an app service if you
56:07
you can set it as an app service if you find that it's uh let's say uh slower or
56:10
find that it's uh let's say uh slower or
56:10
find that it's uh let's say uh slower or something but
56:11
something but
56:11
something but so far i don't have encountered any any
56:13
so far i don't have encountered any any
56:13
so far i don't have encountered any any issues i think it's a
56:16
issues i think it's a
56:16
issues i think it's a uh effective way to
56:19
uh effective way to
56:19
uh effective way to connect to your database with cloud
56:21
connect to your database with cloud
56:21
connect to your database with cloud applications
56:22
applications
56:22
applications great so if i'm understanding correctly
56:24
great so if i'm understanding correctly
56:24
great so if i'm understanding correctly it sounds like we're keeping the
56:25
it sounds like we're keeping the
56:25
it sounds like we're keeping the connection
56:26
connection
56:26
connection um live so that you're always uh
56:30
um live so that you're always uh
56:30
um live so that you're always uh getting the most throughput possible
56:32
getting the most throughput possible
56:32
getting the most throughput possible yeah yeah yeah yeah it's awesome
56:37
i have some more questions uh unless any
56:39
i have some more questions uh unless any
56:39
i have some more questions uh unless any uh leslie or
56:40
uh leslie or
56:40
uh leslie or stephanie if you have any other
56:41
stephanie if you have any other
56:41
stephanie if you have any other questions i can follow some more
56:45
questions i can follow some more
56:45
questions i can follow some more no go for it cool um so uh
56:49
no go for it cool um so uh
56:49
no go for it cool um so uh one question that i actually personally
56:51
one question that i actually personally
56:51
one question that i actually personally was was wondering
56:53
was was wondering
56:53
was was wondering what uh was um what are the best
56:56
what uh was um what are the best
56:56
what uh was um what are the best examples that you would say
56:58
examples that you would say
56:58
examples that you would say for each of the three cases that you
57:00
for each of the three cases that you
57:00
for each of the three cases that you mentioned around
57:01
mentioned around
57:01
mentioned around um you know measuring changes uh input
57:04
um you know measuring changes uh input
57:04
um you know measuring changes uh input bindings and
57:05
bindings and
57:05
bindings and um and out uh output as well like what
57:09
um and out uh output as well like what
57:09
um and out uh output as well like what uh examples have you seen in the wild in
57:10
uh examples have you seen in the wild in
57:10
uh examples have you seen in the wild in terms of using that
57:12
terms of using that
57:12
terms of using that yes for instance well
57:16
yes for instance well
57:16
yes for instance well i i have created a boat that is
57:20
i i have created a boat that is
57:20
i i have created a boat that is running in um actually in
57:23
running in um actually in
57:23
running in um actually in let's say some channel it's youtube
57:27
let's say some channel it's youtube
57:27
let's say some channel it's youtube actually
57:28
actually
57:28
actually and the idea there is that the users
57:31
and the idea there is that the users
57:31
and the idea there is that the users send a request
57:32
send a request
57:32
send a request um and they let's say request music like
57:35
um and they let's say request music like
57:35
um and they let's say request music like okay
57:36
okay
57:36
okay this streaming plays um some music the
57:40
this streaming plays um some music the
57:40
this streaming plays um some music the user
57:40
user
57:40
user wants to listen to some song so it
57:43
wants to listen to some song so it
57:43
wants to listen to some song so it actually uses
57:44
actually uses
57:44
actually uses azure functions i have a boat and this
57:47
azure functions i have a boat and this
57:47
azure functions i have a boat and this bot
57:48
bot
57:48
bot uh calls these these functions the
57:50
uh calls these these functions the
57:50
uh calls these these functions the information is
57:51
information is
57:51
information is saved into a table api
57:54
saved into a table api
57:54
saved into a table api uh sorry table is uh it uses the table
57:57
uh sorry table is uh it uses the table
57:57
uh sorry table is uh it uses the table api of cosmos db
57:59
api of cosmos db
57:59
api of cosmos db and uh gets the information okay this is
58:01
and uh gets the information okay this is
58:01
and uh gets the information okay this is your song it will play next i i
58:03
your song it will play next i i
58:03
your song it will play next i i store it in a queue actually i use a
58:05
store it in a queue actually i use a
58:05
store it in a queue actually i use a queue for for that
58:07
queue for for that
58:07
queue for for that uh so yeah that's one example that i
58:09
uh so yeah that's one example that i
58:09
uh so yeah that's one example that i have been working on
58:10
have been working on
58:10
have been working on i have also seen integrations with iot
58:13
i have also seen integrations with iot
58:13
i have also seen integrations with iot solutions
58:14
solutions
58:14
solutions so in this case you know iot devices
58:17
so in this case you know iot devices
58:17
so in this case you know iot devices generate a lot of information
58:18
generate a lot of information
58:18
generate a lot of information every second every short period of time
58:22
every second every short period of time
58:22
every second every short period of time so and you may you might have
58:25
so and you may you might have
58:25
so and you may you might have distributed your devices in different
58:27
distributed your devices in different
58:27
distributed your devices in different regions in different countries
58:28
regions in different countries
58:28
regions in different countries so you can also send the information to
58:32
so you can also send the information to
58:32
so you can also send the information to the iot hub
58:33
the iot hub
58:33
the iot hub and then process it with the nature
58:35
and then process it with the nature
58:35
and then process it with the nature functions um
58:36
functions um
58:36
functions um analyze your data then store it in
58:39
analyze your data then store it in
58:39
analyze your data then store it in cosmos tv for instance uh
58:41
cosmos tv for instance uh
58:41
cosmos tv for instance uh so yeah that's one another what's
58:43
so yeah that's one another what's
58:43
so yeah that's one another what's another scenario that i have seen
58:46
another scenario that i have seen
58:46
another scenario that i have seen regarding this um yeah that's really
58:49
regarding this um yeah that's really
58:49
regarding this um yeah that's really cool
58:50
cool
58:50
cool um is this music app something you're
58:52
um is this music app something you're
58:52
um is this music app something you're planning on releasing or is this
58:53
planning on releasing or is this
58:53
planning on releasing or is this uh just a side project for your it's a
58:57
uh just a side project for your it's a
58:57
uh just a side project for your it's a it's a project for a friend actually it
59:00
it's a project for a friend actually it
59:00
it's a project for a friend actually it is already
59:01
is already
59:01
is already running live i can share the the link
59:03
running live i can share the the link
59:03
running live i can share the the link later
59:04
later
59:04
later if you would be interested uh it's it's
59:07
if you would be interested uh it's it's
59:07
if you would be interested uh it's it's actually for fun i made it for fun
59:09
actually for fun i made it for fun
59:09
actually for fun i made it for fun i was uh let's say
59:13
i was uh let's say
59:13
i was uh let's say i was a fan of this channel i was
59:15
i was a fan of this channel i was
59:15
i was a fan of this channel i was interested okay
59:16
interested okay
59:16
interested okay can i do it better can i learn actually
59:18
can i do it better can i learn actually
59:18
can i do it better can i learn actually i learned
59:19
i learned
59:19
i learned a lot about functions about integration
59:22
a lot about functions about integration
59:22
a lot about functions about integration with
59:23
with
59:23
with technologies by doing this it was just
59:25
technologies by doing this it was just
59:25
technologies by doing this it was just by fun and
59:26
by fun and
59:26
by fun and yeah that's i think that's where you can
59:29
yeah that's i think that's where you can
59:29
yeah that's i think that's where you can get the most about
59:30
get the most about
59:30
get the most about technology so yeah it's already like
59:33
technology so yeah it's already like
59:33
technology so yeah it's already like actually
59:34
actually
59:34
actually i'm going to share that with you that's
59:36
i'm going to share that with you that's
59:36
i'm going to share that with you that's a really great point i feel like
59:37
a really great point i feel like
59:37
a really great point i feel like combining
59:38
combining
59:38
combining uh something you're passionate about
59:40
uh something you're passionate about
59:40
uh something you're passionate about about something you know to learn more
59:41
about something you know to learn more
59:42
about something you know to learn more about really
59:42
about really
59:42
about really allows you to learn and have fun while
59:44
allows you to learn and have fun while
59:44
allows you to learn and have fun while you're doing it um
59:46
you're doing it um
59:46
you're doing it um and obviously we love when that involves
59:47
and obviously we love when that involves
59:47
and obviously we love when that involves azure cosmos db so i'm definitely going
59:49
azure cosmos db so i'm definitely going
59:49
azure cosmos db so i'm definitely going to mention that
59:50
to mention that
59:50
to mention that but yeah anyways uh that that's really
59:53
but yeah anyways uh that that's really
59:53
but yeah anyways uh that that's really cool and
59:54
cool and
59:54
cool and it was really interesting to learn about
59:55
it was really interesting to learn about
59:55
it was really interesting to learn about azure functions um so
59:57
azure functions um so
59:57
azure functions um so thank you so much for um going through
59:59
thank you so much for um going through
59:59
thank you so much for um going through all of that and
1:00:00
all of that and
1:00:00
all of that and and showing us more about that yeah
1:00:02
and showing us more about that yeah
1:00:02
and showing us more about that yeah thank you for having me
1:00:04
thank you for having me
1:00:04
thank you for having me thank you lewis awesome uh it's a great
1:00:06
thank you lewis awesome uh it's a great
1:00:06
thank you lewis awesome uh it's a great session on functions and how to use them
1:00:08
session on functions and how to use them
1:00:08
session on functions and how to use them with cosmos db
1:00:09
with cosmos db
1:00:09
with cosmos db um so it looks like we're on time so
1:00:11
um so it looks like we're on time so
1:00:11
um so it looks like we're on time so thank you again lewis um
1:00:13
thank you again lewis um
1:00:13
thank you again lewis um so now uh for the next session we have
1:00:16
so now uh for the next session we have
1:00:16
so now uh for the next session we have um
1:00:17
um
1:00:17
um pascal demare uh who's going to do an
1:00:20
pascal demare uh who's going to do an
1:00:20
pascal demare uh who's going to do an awesome question on
1:00:21
awesome question on
1:00:21
awesome question on uh later modeling and new design
1:00:25
uh later modeling and new design
1:00:25
uh later modeling and new design for cosmos db uh take it away pascal
1:00:29
for cosmos db uh take it away pascal
1:00:29
for cosmos db uh take it away pascal thank you theo hello everyone and
1:00:32
thank you theo hello everyone and
1:00:32
thank you theo hello everyone and welcome
1:00:32
welcome
1:00:32
welcome from brussels belgium for this session
1:00:35
from brussels belgium for this session
1:00:35
from brussels belgium for this session of the cosmos db conference
1:00:39
of the cosmos db conference
1:00:39
of the cosmos db conference so let's say that you're new to cosmos
1:00:41
so let's say that you're new to cosmos
1:00:41
so let's say that you're new to cosmos db
1:00:42
db
1:00:42
db but you already understand all of the
1:00:44
but you already understand all of the
1:00:44
but you already understand all of the great things that it can bring you
1:00:46
great things that it can bring you
1:00:46
great things that it can bring you with high availability high throughput
1:00:49
with high availability high throughput
1:00:49
with high availability high throughput global distribution
1:00:51
global distribution
1:00:51
global distribution now you decide that it's time to give it
1:00:53
now you decide that it's time to give it
1:00:53
now you decide that it's time to give it a try
1:00:54
a try
1:00:54
a try you go to the azure console you create a
1:00:57
you go to the azure console you create a
1:00:57
you go to the azure console you create a cosmos db instance you select the
1:01:00
cosmos db instance you select the
1:01:00
cosmos db instance you select the resource
1:01:01
resource
1:01:01
resource group and then create an account name
1:01:05
group and then create an account name
1:01:05
group and then create an account name and then you get to this third question
1:01:08
and then you get to this third question
1:01:08
and then you get to this third question choose the api
1:01:09
choose the api
1:01:09
choose the api and data model to use with this account
1:01:13
and data model to use with this account
1:01:13
and data model to use with this account along with a drop down with five
1:01:15
along with a drop down with five
1:01:15
along with a drop down with five possible options
1:01:17
possible options
1:01:17
possible options so which one do you choose
1:01:20
so which one do you choose
1:01:20
so which one do you choose why does it say core sql i thought
1:01:23
why does it say core sql i thought
1:01:24
why does it say core sql i thought this was a nosql database so in this
1:01:27
this was a nosql database so in this
1:01:27
this was a nosql database so in this session
1:01:27
session
1:01:27
session we will attempt to clarify why the
1:01:30
we will attempt to clarify why the
1:01:30
we will attempt to clarify why the choice of this question is critical
1:01:33
choice of this question is critical
1:01:33
choice of this question is critical and how to choose the best option to
1:01:35
and how to choose the best option to
1:01:35
and how to choose the best option to suit your needs
1:01:37
suit your needs
1:01:37
suit your needs so let me introduce myself i'm pascal
1:01:39
so let me introduce myself i'm pascal
1:01:39
so let me introduce myself i'm pascal demare
1:01:40
demare
1:01:40
demare founder and ceo of accolade we produce
1:01:44
founder and ceo of accolade we produce
1:01:44
founder and ceo of accolade we produce a data modeling software for nosql
1:01:46
a data modeling software for nosql
1:01:46
a data modeling software for nosql databases in general and cosmos db
1:01:49
databases in general and cosmos db
1:01:49
databases in general and cosmos db in particular we developed what may look
1:01:52
in particular we developed what may look
1:01:52
in particular we developed what may look like
1:01:53
like
1:01:53
like a traditional entity relationship
1:01:55
a traditional entity relationship
1:01:55
a traditional entity relationship diagram
1:01:56
diagram
1:01:56
diagram uh modeling tool except that it was
1:01:59
uh modeling tool except that it was
1:01:59
uh modeling tool except that it was specifically designed to handle the
1:02:02
specifically designed to handle the
1:02:02
specifically designed to handle the things that
1:02:03
things that
1:02:03
things that the traditional tools could not handle
1:02:05
the traditional tools could not handle
1:02:05
the traditional tools could not handle and that is
1:02:06
and that is
1:02:06
and that is the complex structures found in modern
1:02:09
the complex structures found in modern
1:02:09
the complex structures found in modern big data
1:02:10
big data
1:02:10
big data and also property graph databases like
1:02:13
and also property graph databases like
1:02:14
and also property graph databases like the gremlin api enables so
1:02:17
the gremlin api enables so
1:02:18
the gremlin api enables so nosql databases are usually divided in
1:02:21
nosql databases are usually divided in
1:02:21
nosql databases are usually divided in uh four families and cosmos db
1:02:24
uh four families and cosmos db
1:02:24
uh four families and cosmos db supports all of these different uh
1:02:26
supports all of these different uh
1:02:26
supports all of these different uh flavors
1:02:28
flavors
1:02:28
flavors json documents stores
1:02:31
json documents stores
1:02:31
json documents stores with um that are supported
1:02:34
with um that are supported
1:02:34
with um that are supported with two different apis the sql-like
1:02:39
with two different apis the sql-like
1:02:39
with two different apis the sql-like api called core and also the mongodb api
1:02:46
and uh a key value store with the table
1:02:49
and uh a key value store with the table
1:02:49
and uh a key value store with the table api
1:02:50
api
1:02:50
api the property graph uh database
1:02:53
the property graph uh database
1:02:53
the property graph uh database accessed via the gremlin api
1:02:56
accessed via the gremlin api
1:02:56
accessed via the gremlin api and uh the cassandra api for column
1:03:00
and uh the cassandra api for column
1:03:00
and uh the cassandra api for column families
1:03:02
families
1:03:02
families cosmos db is unique in allowing
1:03:05
cosmos db is unique in allowing
1:03:05
cosmos db is unique in allowing something that was labeled polyglot
1:03:08
something that was labeled polyglot
1:03:08
something that was labeled polyglot persistence
1:03:09
persistence
1:03:09
persistence by martin fowler back in 2012.
1:03:13
by martin fowler back in 2012.
1:03:13
by martin fowler back in 2012. he said that instead of just picking a
1:03:16
he said that instead of just picking a
1:03:16
he said that instead of just picking a relational database
1:03:21
because everybody does we need to
1:03:23
because everybody does we need to
1:03:23
because everybody does we need to understand
1:03:25
understand
1:03:25
understand the nature of the data that we're
1:03:27
the nature of the data that we're
1:03:27
the nature of the data that we're storing and how
1:03:28
storing and how
1:03:28
storing and how we want to manipulate it the result is
1:03:31
we want to manipulate it the result is
1:03:31
we want to manipulate it the result is that
1:03:32
that
1:03:32
that most organizations will have a mix of
1:03:34
most organizations will have a mix of
1:03:34
most organizations will have a mix of data storage technologies for different
1:03:37
data storage technologies for different
1:03:37
data storage technologies for different circumstances
1:03:38
circumstances
1:03:38
circumstances and that's critical because um
1:03:42
and that's critical because um
1:03:42
and that's critical because um in we need to realize that applications
1:03:46
in we need to realize that applications
1:03:46
in we need to realize that applications quickly become very complex and it would
1:03:49
quickly become very complex and it would
1:03:49
quickly become very complex and it would be an
1:03:50
be an
1:03:50
be an illusion to think that there is a single
1:03:53
illusion to think that there is a single
1:03:53
illusion to think that there is a single storage model that
1:03:55
storage model that
1:03:55
storage model that felt fits all purposes so choosing the
1:03:58
felt fits all purposes so choosing the
1:03:58
felt fits all purposes so choosing the best tool for each use case
1:04:01
best tool for each use case
1:04:01
best tool for each use case ensures that you don't have to make
1:04:03
ensures that you don't have to make
1:04:03
ensures that you don't have to make endless compromises
1:04:05
endless compromises
1:04:05
endless compromises and that you can leverage the strength
1:04:08
and that you can leverage the strength
1:04:08
and that you can leverage the strength of
1:04:08
of
1:04:08
of each storage model
1:04:12
each storage model
1:04:12
each storage model so if we take um an example of
1:04:16
so if we take um an example of
1:04:16
so if we take um an example of an e-commerce platform for example
1:04:19
an e-commerce platform for example
1:04:19
an e-commerce platform for example we can see that each section of the user
1:04:22
we can see that each section of the user
1:04:22
we can see that each section of the user experience
1:04:23
experience
1:04:23
experience and processing is actually very
1:04:26
and processing is actually very
1:04:26
and processing is actually very different
1:04:27
different
1:04:28
different while you your master data may be stored
1:04:31
while you your master data may be stored
1:04:31
while you your master data may be stored in a relational database like azure sql
1:04:36
in a relational database like azure sql
1:04:36
in a relational database like azure sql you may want to have uh used the
1:04:39
you may want to have uh used the
1:04:39
you may want to have uh used the cosmos db table api to store
1:04:43
cosmos db table api to store
1:04:43
cosmos db table api to store very quickly and be able to retrieve
1:04:46
very quickly and be able to retrieve
1:04:46
very quickly and be able to retrieve shopping cart and session data
1:04:50
then completed orders as well as serving
1:04:54
then completed orders as well as serving
1:04:54
then completed orders as well as serving the product catalog
1:04:56
the product catalog
1:04:56
the product catalog for shoppers are would be best served by
1:05:00
for shoppers are would be best served by
1:05:00
for shoppers are would be best served by the cosmos db json document
1:05:03
the cosmos db json document
1:05:03
the cosmos db json document apis either the sql api or the mongodb
1:05:06
apis either the sql api or the mongodb
1:05:06
apis either the sql api or the mongodb api logs data
1:05:09
api logs data
1:05:09
api logs data on the other hand on which you may be
1:05:11
on the other hand on which you may be
1:05:12
on the other hand on which you may be doing aggregation for specific
1:05:15
doing aggregation for specific
1:05:15
doing aggregation for specific specific columns would be best served
1:05:18
specific columns would be best served
1:05:18
specific columns would be best served through the cassandra
1:05:19
through the cassandra
1:05:20
through the cassandra api and then you've got fraud detection
1:05:24
api and then you've got fraud detection
1:05:24
api and then you've got fraud detection during checkout or you may have a
1:05:26
during checkout or you may have a
1:05:26
during checkout or you may have a recommendation engine
1:05:28
recommendation engine
1:05:28
recommendation engine and that's where a property graph
1:05:30
and that's where a property graph
1:05:30
and that's where a property graph database will shine
1:05:32
database will shine
1:05:32
database will shine and so the gremlin api will be best for
1:05:36
and so the gremlin api will be best for
1:05:36
and so the gremlin api will be best for that use case
1:05:37
that use case
1:05:37
that use case and then finally you might have
1:05:40
and then finally you might have
1:05:40
and then finally you might have an analytics store for uh
1:05:43
an analytics store for uh
1:05:43
an analytics store for uh you know data self-service
1:05:47
you know data self-service
1:05:47
you know data self-service and for that azure synapse would be your
1:05:50
and for that azure synapse would be your
1:05:50
and for that azure synapse would be your ideal
1:05:51
ideal
1:05:51
ideal tool to serve this use case
1:05:55
tool to serve this use case
1:05:55
tool to serve this use case um there's other examples
1:05:58
um there's other examples
1:05:58
um there's other examples of you know where polyglot persistence
1:06:01
of you know where polyglot persistence
1:06:01
of you know where polyglot persistence is in place and for example a data
1:06:05
is in place and for example a data
1:06:05
is in place and for example a data pipeline from transactional
1:06:08
pipeline from transactional
1:06:08
pipeline from transactional databases through analytics and
1:06:11
databases through analytics and
1:06:11
databases through analytics and self-service
1:06:12
self-service
1:06:12
self-service data democratization so
1:06:16
data democratization so
1:06:16
data democratization so but this takes us off the subject of
1:06:18
but this takes us off the subject of
1:06:18
but this takes us off the subject of cosmos db but it sure
1:06:20
cosmos db but it sure
1:06:20
cosmos db but it sure it goes to show how uh polyglot
1:06:23
it goes to show how uh polyglot
1:06:23
it goes to show how uh polyglot persistence
1:06:24
persistence
1:06:24
persistence is a reality and we need to use the
1:06:27
is a reality and we need to use the
1:06:28
is a reality and we need to use the right tool for the job
1:06:30
right tool for the job
1:06:30
right tool for the job so um in this table
1:06:33
so um in this table
1:06:33
so um in this table um which is a bit of a caricature we
1:06:36
um which is a bit of a caricature we
1:06:36
um which is a bit of a caricature we have
1:06:37
have
1:06:37
have a comparison of all of these uh
1:06:39
a comparison of all of these uh
1:06:39
a comparison of all of these uh technologies
1:06:40
technologies
1:06:40
technologies and uh what makes
1:06:43
and uh what makes
1:06:43
and uh what makes one better to the other uh better than
1:06:46
one better to the other uh better than
1:06:46
one better to the other uh better than the other
1:06:47
the other
1:06:47
the other on specific use cases but it goes to
1:06:51
on specific use cases but it goes to
1:06:51
on specific use cases but it goes to show also how one technology
1:06:55
show also how one technology
1:06:55
show also how one technology cannot fit all and is
1:06:58
cannot fit all and is
1:06:58
cannot fit all and is not you know in absolute terms better
1:07:01
not you know in absolute terms better
1:07:01
not you know in absolute terms better than the other technologies
1:07:03
than the other technologies
1:07:03
than the other technologies and so we see here you know as we saw in
1:07:06
and so we see here you know as we saw in
1:07:06
and so we see here you know as we saw in the example
1:07:07
the example
1:07:07
the example earlier that relational best fits
1:07:11
earlier that relational best fits
1:07:11
earlier that relational best fits use cases of master data and
1:07:14
use cases of master data and
1:07:14
use cases of master data and transactions
1:07:15
transactions
1:07:15
transactions it has the benefits of being well known
1:07:18
it has the benefits of being well known
1:07:18
it has the benefits of being well known by
1:07:19
by
1:07:19
by you know many people and enforcing the
1:07:22
you know many people and enforcing the
1:07:22
you know many people and enforcing the schema
1:07:23
schema
1:07:23
schema at the engine level but it comes with
1:07:27
at the engine level but it comes with
1:07:27
at the engine level but it comes with problems such as the fact that schemas
1:07:29
problems such as the fact that schemas
1:07:29
problems such as the fact that schemas are hard to change
1:07:31
are hard to change
1:07:31
are hard to change and the technology does not scale well
1:07:34
and the technology does not scale well
1:07:34
and the technology does not scale well or
1:07:34
or
1:07:34
or horizontally so key value stores are
1:07:37
horizontally so key value stores are
1:07:37
horizontally so key value stores are great for caching
1:07:41
and so incredibly fast performance for
1:07:45
and so incredibly fast performance for
1:07:45
and so incredibly fast performance for reads and writes
1:07:46
reads and writes
1:07:46
reads and writes but if you're doing queries on anything
1:07:49
but if you're doing queries on anything
1:07:49
but if you're doing queries on anything else
1:07:50
else
1:07:50
else then the the key um it's going to be
1:07:53
then the the key um it's going to be
1:07:53
then the the key um it's going to be really difficult if not impossible
1:07:55
really difficult if not impossible
1:07:55
really difficult if not impossible that's not what it was done
1:07:57
that's not what it was done
1:07:57
that's not what it was done for but you know as we could see there's
1:08:00
for but you know as we could see there's
1:08:00
for but you know as we could see there's a specific use cases where it shines
1:08:03
a specific use cases where it shines
1:08:03
a specific use cases where it shines document databases on the other hand
1:08:06
document databases on the other hand
1:08:06
document databases on the other hand are extremely still
1:08:10
are extremely still
1:08:10
are extremely still very performant in terms of
1:08:14
very performant in terms of
1:08:14
very performant in terms of access speed it
1:08:18
access speed it
1:08:18
access speed it allows or even encourages
1:08:21
allows or even encourages
1:08:21
allows or even encourages data duplication which some might see as
1:08:24
data duplication which some might see as
1:08:24
data duplication which some might see as a challenge its high flexibility
1:08:29
a challenge its high flexibility
1:08:29
a challenge its high flexibility is um presents the risk that
1:08:32
is um presents the risk that
1:08:32
is um presents the risk that people might use it as a dumping ground
1:08:34
people might use it as a dumping ground
1:08:34
people might use it as a dumping ground so um
1:08:35
so um
1:08:36
so um you know you need to be watching for
1:08:37
you know you need to be watching for
1:08:37
you know you need to be watching for that columnar databases
1:08:40
that columnar databases
1:08:40
that columnar databases um you know served by the cassandra api
1:08:43
um you know served by the cassandra api
1:08:43
um you know served by the cassandra api it's really great for sparse data and it
1:08:47
it's really great for sparse data and it
1:08:47
it's really great for sparse data and it has very fast column operations
1:08:51
has very fast column operations
1:08:51
has very fast column operations but it's really not ideal if your data
1:08:54
but it's really not ideal if your data
1:08:54
but it's really not ideal if your data is not
1:08:55
is not
1:08:55
is not immutable graph databases are
1:08:59
immutable graph databases are
1:08:59
immutable graph databases are you know a separate family it's great
1:09:02
you know a separate family it's great
1:09:02
you know a separate family it's great for
1:09:03
for
1:09:03
for finding hidden relationships
1:09:05
finding hidden relationships
1:09:05
finding hidden relationships relationships that didn't
1:09:07
relationships that didn't
1:09:07
relationships that didn't that you didn't know existed when you
1:09:09
that you didn't know existed when you
1:09:09
that you didn't know existed when you did your design
1:09:11
did your design
1:09:11
did your design but that you can figure out through
1:09:13
but that you can figure out through
1:09:13
but that you can figure out through traversal
1:09:14
traversal
1:09:14
traversal um of the graph
1:09:18
um of the graph
1:09:18
um of the graph through your queries but it can be a bit
1:09:21
through your queries but it can be a bit
1:09:21
through your queries but it can be a bit difficult to
1:09:22
difficult to
1:09:22
difficult to uh model and data warehouse like synapse
1:09:26
uh model and data warehouse like synapse
1:09:26
uh model and data warehouse like synapse help you slice and dice the data for
1:09:30
help you slice and dice the data for
1:09:30
help you slice and dice the data for analytics
1:09:31
analytics
1:09:31
analytics but of course it's not for transactional
1:09:35
but of course it's not for transactional
1:09:35
but of course it's not for transactional use cases so
1:09:39
use cases so
1:09:39
use cases so transitioning to the challenges when
1:09:41
transitioning to the challenges when
1:09:41
transitioning to the challenges when doing data modeling and schema and
1:09:43
doing data modeling and schema and
1:09:43
doing data modeling and schema and design for cosmos db
1:09:46
design for cosmos db
1:09:46
design for cosmos db you may have heard people say that with
1:09:48
you may have heard people say that with
1:09:48
you may have heard people say that with schema-less databases
1:09:50
schema-less databases
1:09:50
schema-less databases you don't need to care about the schema
1:09:53
you don't need to care about the schema
1:09:53
you don't need to care about the schema they also call this a schema on read
1:09:57
they also call this a schema on read
1:09:57
they also call this a schema on read meaning that the schema does not need to
1:09:59
meaning that the schema does not need to
1:09:59
meaning that the schema does not need to be declared ahead of time
1:10:01
be declared ahead of time
1:10:01
be declared ahead of time and can be figured figured out when you
1:10:03
and can be figured figured out when you
1:10:04
and can be figured figured out when you read the data
1:10:05
read the data
1:10:05
read the data but these terms can be misleading
1:10:07
but these terms can be misleading
1:10:08
but these terms can be misleading because in effect
1:10:09
because in effect
1:10:09
because in effect when you persist data
1:10:12
when you persist data
1:10:12
when you persist data it automatically has a schema it has a
1:10:15
it automatically has a schema it has a
1:10:15
it automatically has a schema it has a structure
1:10:17
structure
1:10:17
structure inherent to the fact that you just
1:10:19
inherent to the fact that you just
1:10:19
inherent to the fact that you just stored it
1:10:20
stored it
1:10:20
stored it so we prefer to talk about dynamic
1:10:23
so we prefer to talk about dynamic
1:10:23
so we prefer to talk about dynamic schema evolution particularly
1:10:25
schema evolution particularly
1:10:25
schema evolution particularly in an agile environment when customers
1:10:29
in an agile environment when customers
1:10:29
in an agile environment when customers needs evolve constantly and at an
1:10:32
needs evolve constantly and at an
1:10:32
needs evolve constantly and at an increasingly fast pace
1:10:34
increasingly fast pace
1:10:34
increasingly fast pace so uh be careful uh not to confuse
1:10:38
so uh be careful uh not to confuse
1:10:38
so uh be careful uh not to confuse flexibility with sloppiness and don't
1:10:40
flexibility with sloppiness and don't
1:10:40
flexibility with sloppiness and don't take this
1:10:41
take this
1:10:42
take this from me gardner has been alerting
1:10:44
from me gardner has been alerting
1:10:44
from me gardner has been alerting customers
1:10:45
customers
1:10:45
customers not to fall into this trap and uh
1:10:48
not to fall into this trap and uh
1:10:48
not to fall into this trap and uh approach this with sober awareness
1:10:51
approach this with sober awareness
1:10:51
approach this with sober awareness to the fact that um you know schema and
1:10:55
to the fact that um you know schema and
1:10:55
to the fact that um you know schema and read
1:10:55
read
1:10:55
read does not mean scheme by amateurs
1:11:00
does not mean scheme by amateurs
1:11:00
does not mean scheme by amateurs so when doing the design for schema on
1:11:03
so when doing the design for schema on
1:11:03
so when doing the design for schema on read or schema less database
1:11:06
read or schema less database
1:11:06
read or schema less database it's really critical to forget what
1:11:09
it's really critical to forget what
1:11:09
it's really critical to forget what we've learned for decades with
1:11:11
we've learned for decades with
1:11:11
we've learned for decades with relational databases
1:11:13
relational databases
1:11:13
relational databases the rules of normalizations have taught
1:11:16
the rules of normalizations have taught
1:11:16
the rules of normalizations have taught us to design
1:11:17
us to design
1:11:17
us to design data models in an application agnostic
1:11:20
data models in an application agnostic
1:11:20
data models in an application agnostic manner
1:11:21
manner
1:11:21
manner making sure to have this beautiful
1:11:24
making sure to have this beautiful
1:11:24
making sure to have this beautiful generic design
1:11:26
generic design
1:11:26
generic design with no duplication of data
1:11:30
with no duplication of data
1:11:30
with no duplication of data so when it comes to nosql databases
1:11:35
so when it comes to nosql databases
1:11:35
so when it comes to nosql databases we need to get rid of all this baggage
1:11:37
we need to get rid of all this baggage
1:11:37
we need to get rid of all this baggage we need to turn the rules
1:11:39
we need to turn the rules
1:11:39
we need to turn the rules upside down based on the requirements
1:11:43
upside down based on the requirements
1:11:43
upside down based on the requirements and the design of the application
1:11:45
and the design of the application
1:11:45
and the design of the application we need to identify
1:11:48
we need to identify
1:11:48
we need to identify and analyze the access patterns the
1:11:52
and analyze the access patterns the
1:11:52
and analyze the access patterns the actual queries needed to access the data
1:11:56
actual queries needed to access the data
1:11:56
actual queries needed to access the data uh whether it's for screen display or
1:11:58
uh whether it's for screen display or
1:11:58
uh whether it's for screen display or reports or download
1:12:01
reports or download
1:12:01
reports or download sorry or down stream processing you know
1:12:04
sorry or down stream processing you know
1:12:04
sorry or down stream processing you know these
1:12:04
these
1:12:04
these access patterns are critical so we can
1:12:07
access patterns are critical so we can
1:12:08
access patterns are critical so we can design the data model
1:12:11
design the data model
1:12:11
design the data model in a way to satisfy these queries in the
1:12:13
in a way to satisfy these queries in the
1:12:13
in a way to satisfy these queries in the most efficient manner
1:12:16
most efficient manner
1:12:16
most efficient manner which is by avoiding
1:12:19
which is by avoiding
1:12:19
which is by avoiding joints that inevitably slow down
1:12:21
joints that inevitably slow down
1:12:21
joints that inevitably slow down performance and that will dictate
1:12:24
performance and that will dictate
1:12:24
performance and that will dictate how we store the data okay so
1:12:27
how we store the data okay so
1:12:27
how we store the data okay so we we will first figure out how we will
1:12:30
we we will first figure out how we will
1:12:30
we we will first figure out how we will read the data
1:12:32
read the data
1:12:32
read the data before we figure out how we store the
1:12:34
before we figure out how we store the
1:12:34
before we figure out how we store the data
1:12:36
data
1:12:36
data so i strongly suggest
1:12:39
so i strongly suggest
1:12:39
so i strongly suggest that one should never write a line of
1:12:43
that one should never write a line of
1:12:43
that one should never write a line of code
1:12:44
code
1:12:44
code without truly understanding
1:12:47
without truly understanding
1:12:47
without truly understanding the design of the application and
1:12:50
the design of the application and
1:12:50
the design of the application and hence the data model and then in an
1:12:53
hence the data model and then in an
1:12:53
hence the data model and then in an agile manner you can
1:12:55
agile manner you can
1:12:55
agile manner you can iterate uh on this flexible schema
1:12:59
iterate uh on this flexible schema
1:12:59
iterate uh on this flexible schema and the application that leverages it
1:13:02
and the application that leverages it
1:13:02
and the application that leverages it and that's the really
1:13:03
and that's the really
1:13:03
and that's the really agile way of doing things right
1:13:08
so a data model is
1:13:11
so a data model is
1:13:11
so a data model is the blueprint for applications um you
1:13:14
the blueprint for applications um you
1:13:14
the blueprint for applications um you would not imagine
1:13:15
would not imagine
1:13:15
would not imagine building a house without a blueprint for
1:13:18
building a house without a blueprint for
1:13:18
building a house without a blueprint for the architect and contractor
1:13:21
the architect and contractor
1:13:21
the architect and contractor to interact and communicate correct
1:13:24
to interact and communicate correct
1:13:24
to interact and communicate correct a data model allows two things it allows
1:13:28
a data model allows two things it allows
1:13:28
a data model allows two things it allows for the architect to rigorously think
1:13:31
for the architect to rigorously think
1:13:31
for the architect to rigorously think through
1:13:32
through
1:13:32
through all of the details of his originally
1:13:35
all of the details of his originally
1:13:35
all of the details of his originally sketchy vision
1:13:36
sketchy vision
1:13:36
sketchy vision and as a communication tool it helps
1:13:40
and as a communication tool it helps
1:13:40
and as a communication tool it helps discuss his vision with
1:13:43
discuss his vision with
1:13:43
discuss his vision with the owner on one hand uh
1:13:47
the owner on one hand uh
1:13:47
the owner on one hand uh to make sure that you know we all agree
1:13:49
to make sure that you know we all agree
1:13:49
to make sure that you know we all agree on the vision
1:13:51
on the vision
1:13:51
on the vision and also with the building contractors
1:13:53
and also with the building contractors
1:13:53
and also with the building contractors and the builders to make sure that they
1:13:56
and the builders to make sure that they
1:13:56
and the builders to make sure that they will execute
1:13:57
will execute
1:13:57
will execute uh on that vision properly
1:14:01
uh on that vision properly
1:14:01
uh on that vision properly so while you might not need an
1:14:03
so while you might not need an
1:14:03
so while you might not need an architectural blueprint
1:14:05
architectural blueprint
1:14:05
architectural blueprint to build a shack in the backyard you
1:14:08
to build a shack in the backyard you
1:14:08
to build a shack in the backyard you will definitely need one to build a
1:14:10
will definitely need one to build a
1:14:10
will definitely need one to build a house
1:14:11
house
1:14:11
house or skyscraper if you want to avoid
1:14:14
or skyscraper if you want to avoid
1:14:14
or skyscraper if you want to avoid huge technical debt you'd better think
1:14:17
huge technical debt you'd better think
1:14:17
huge technical debt you'd better think think things through before you write a
1:14:20
think things through before you write a
1:14:20
think things through before you write a line of code
1:14:22
line of code
1:14:22
line of code so yesterday in the conference there was
1:14:25
so yesterday in the conference there was
1:14:25
so yesterday in the conference there was a
1:14:25
a
1:14:25
a fantastic session by uh
1:14:28
fantastic session by uh
1:14:28
fantastic session by uh lenny lobel on all the
1:14:32
lenny lobel on all the
1:14:32
lenny lobel on all the on this important subject of data
1:14:34
on this important subject of data
1:14:34
on this important subject of data modeling and partitioning for cosmos db
1:14:38
modeling and partitioning for cosmos db
1:14:38
modeling and partitioning for cosmos db you may get the video on demand
1:14:41
you may get the video on demand
1:14:41
you may get the video on demand and i'm not going to repeat here what
1:14:43
and i'm not going to repeat here what
1:14:43
and i'm not going to repeat here what was said during the session
1:14:46
was said during the session
1:14:46
was said during the session the session was mainly on document
1:14:48
the session was mainly on document
1:14:48
the session was mainly on document stores with the core sql api or mongodb
1:14:51
stores with the core sql api or mongodb
1:14:51
stores with the core sql api or mongodb api but i'll reinforce the message
1:14:54
api but i'll reinforce the message
1:14:54
api but i'll reinforce the message and talk also about the grenlin api
1:14:58
and talk also about the grenlin api
1:14:58
and talk also about the grenlin api the beauty of nosql databases is
1:15:02
the beauty of nosql databases is
1:15:02
the beauty of nosql databases is the ability of avoiding
1:15:05
the ability of avoiding
1:15:05
the ability of avoiding the costly joins when reading data
1:15:09
the costly joins when reading data
1:15:09
the costly joins when reading data which brings a much higher performance
1:15:11
which brings a much higher performance
1:15:11
which brings a much higher performance of course if you avoid the joints
1:15:15
how is this achieved actually you're
1:15:18
how is this achieved actually you're
1:15:18
how is this achieved actually you're doing
1:15:19
doing
1:15:19
doing joints but you're doing the joints when
1:15:21
joints but you're doing the joints when
1:15:21
joints but you're doing the joints when you write the data
1:15:23
you write the data
1:15:23
you write the data and that makes it so when you read the
1:15:26
and that makes it so when you read the
1:15:26
and that makes it so when you read the data
1:15:27
data
1:15:27
data access is incredibly fast so
1:15:31
access is incredibly fast so
1:15:31
access is incredibly fast so that sounds great but it should be done
1:15:34
that sounds great but it should be done
1:15:34
that sounds great but it should be done carefully
1:15:35
carefully
1:15:35
carefully with the full understanding of the
1:15:37
with the full understanding of the
1:15:37
with the full understanding of the cardinality
1:15:39
cardinality
1:15:39
cardinality of the relationships of the data you're
1:15:42
of the relationships of the data you're
1:15:42
of the relationships of the data you're suddenly
1:15:43
suddenly
1:15:43
suddenly now embedding together so for example
1:15:48
now embedding together so for example
1:15:48
now embedding together so for example you wouldn't want to have such a high
1:15:51
you wouldn't want to have such a high
1:15:51
you wouldn't want to have such a high cardinality
1:15:53
cardinality
1:15:53
cardinality that the document would grow out of the
1:15:55
that the document would grow out of the
1:15:55
that the document would grow out of the hand
1:15:56
hand
1:15:56
hand so for example if you had a
1:16:00
so for example if you had a
1:16:00
so for example if you had a social network and you have a million
1:16:03
social network and you have a million
1:16:03
social network and you have a million followers
1:16:04
followers
1:16:04
followers you don't want to put the million
1:16:06
you don't want to put the million
1:16:06
you don't want to put the million followers
1:16:07
followers
1:16:07
followers in the same document because this
1:16:09
in the same document because this
1:16:09
in the same document because this document would grow
1:16:11
document would grow
1:16:11
document would grow out of hand so referencing is still a
1:16:15
out of hand so referencing is still a
1:16:15
out of hand so referencing is still a possibility
1:16:16
possibility
1:16:16
possibility with uh nosql and obviously with cosmos
1:16:20
with uh nosql and obviously with cosmos
1:16:20
with uh nosql and obviously with cosmos db
1:16:21
db
1:16:21
db um and it should be used where
1:16:24
um and it should be used where
1:16:24
um and it should be used where applicable
1:16:25
applicable
1:16:25
applicable um the only difference is you don't
1:16:29
um the only difference is you don't
1:16:29
um the only difference is you don't you cannot count on the database engine
1:16:32
you cannot count on the database engine
1:16:32
you cannot count on the database engine to enforce
1:16:33
to enforce
1:16:33
to enforce these relationships and so you need to
1:16:36
these relationships and so you need to
1:16:36
these relationships and so you need to build that
1:16:37
build that
1:16:37
build that in your application so
1:16:41
in your application so
1:16:41
in your application so aside from the important restriction of
1:16:44
aside from the important restriction of
1:16:44
aside from the important restriction of the
1:16:45
the
1:16:45
the cardinality you should do embedding
1:16:50
cardinality you should do embedding
1:16:50
cardinality you should do embedding when in doubt you know favor and betting
1:16:52
when in doubt you know favor and betting
1:16:52
when in doubt you know favor and betting when in doubt
1:16:54
when in doubt
1:16:54
when in doubt the reason is that you can store
1:16:56
the reason is that you can store
1:16:56
the reason is that you can store together
1:16:57
together
1:16:57
together information that belongs together so it
1:16:59
information that belongs together so it
1:17:00
information that belongs together so it makes a lot of sense
1:17:01
makes a lot of sense
1:17:01
makes a lot of sense not only for humans that all of this
1:17:04
not only for humans that all of this
1:17:04
not only for humans that all of this information that
1:17:05
information that
1:17:05
information that belongs together is put together but
1:17:07
belongs together is put together but
1:17:07
belongs together is put together but also
1:17:08
also
1:17:08
also for the uh for the systems because they
1:17:12
for the uh for the systems because they
1:17:12
for the uh for the systems because they only need to access the information in
1:17:15
only need to access the information in
1:17:15
only need to access the information in one place
1:17:16
one place
1:17:16
one place instead of gathering information from
1:17:19
instead of gathering information from
1:17:19
instead of gathering information from all different places when doing joints
1:17:23
all different places when doing joints
1:17:23
all different places when doing joints so um if we take the starting point of
1:17:27
so um if we take the starting point of
1:17:27
so um if we take the starting point of lenny's
1:17:27
lenny's
1:17:27
lenny's lesson less session less yesterday
1:17:31
lesson less session less yesterday
1:17:31
lesson less session less yesterday and we put it into an entity
1:17:33
and we put it into an entity
1:17:33
and we put it into an entity relationship
1:17:34
relationship
1:17:34
relationship diagram with nicely normalized
1:17:37
diagram with nicely normalized
1:17:37
diagram with nicely normalized flat structures as we would store
1:17:42
flat structures as we would store
1:17:42
flat structures as we would store traditionally in a relational database
1:17:44
traditionally in a relational database
1:17:44
traditionally in a relational database we would get
1:17:45
we would get
1:17:45
we would get you know this erd
1:17:49
you know this erd
1:17:49
you know this erd um that we're all familiar with
1:17:52
um that we're all familiar with
1:17:52
um that we're all familiar with and through a process of um
1:17:56
and through a process of um
1:17:56
and through a process of um importing the ddl uh of
1:17:59
importing the ddl uh of
1:17:59
importing the ddl uh of that structure applying the principles
1:18:03
that structure applying the principles
1:18:03
that structure applying the principles of
1:18:04
of
1:18:04
of denormalization as we've seen to
1:18:06
denormalization as we've seen to
1:18:06
denormalization as we've seen to embed data based on the analysis
1:18:10
embed data based on the analysis
1:18:10
embed data based on the analysis of the access patterns and queries
1:18:13
of the access patterns and queries
1:18:13
of the access patterns and queries we come up with a schema design that is
1:18:16
we come up with a schema design that is
1:18:16
we come up with a schema design that is optimized for
1:18:17
optimized for
1:18:17
optimized for cosmos db we can iterate
1:18:21
cosmos db we can iterate
1:18:21
cosmos db we can iterate on the structure after discussing with
1:18:23
on the structure after discussing with
1:18:23
on the structure after discussing with the various
1:18:24
the various
1:18:24
the various stakeholders of the application making
1:18:27
stakeholders of the application making
1:18:27
stakeholders of the application making sure
1:18:28
sure
1:18:28
sure that we achieve the expected objectives
1:18:31
that we achieve the expected objectives
1:18:31
that we achieve the expected objectives then we create a series of artifacts
1:18:35
then we create a series of artifacts
1:18:35
then we create a series of artifacts including documentation schemas and
1:18:38
including documentation schemas and
1:18:38
including documentation schemas and even rest apis and we end up
1:18:42
even rest apis and we end up
1:18:42
even rest apis and we end up with this kind of uh data model
1:18:46
with this kind of uh data model
1:18:46
with this kind of uh data model where you can see how
1:18:49
where you can see how
1:18:49
where you can see how here on the left customer addresses
1:18:52
here on the left customer addresses
1:18:52
here on the left customer addresses have been embedded into the customer
1:18:55
have been embedded into the customer
1:18:55
have been embedded into the customer document
1:18:56
document
1:18:56
document along with password information
1:18:59
along with password information
1:18:59
along with password information on the sales order side we see
1:19:02
on the sales order side we see
1:19:02
on the sales order side we see how the order details uh
1:19:05
how the order details uh
1:19:05
how the order details uh info has been embedded as an array
1:19:08
info has been embedded as an array
1:19:08
info has been embedded as an array of sub-documents and um
1:19:12
of sub-documents and um
1:19:12
of sub-documents and um etc etc and we've gone from
1:19:15
etc etc and we've gone from
1:19:16
etc etc and we've gone from nine tables in the original uh table
1:19:19
nine tables in the original uh table
1:19:19
nine tables in the original uh table uh in the original model relational
1:19:22
uh in the original model relational
1:19:22
uh in the original model relational to just three containers here
1:19:26
to just three containers here
1:19:26
to just three containers here they're separate containers because the
1:19:29
they're separate containers because the
1:19:29
they're separate containers because the partition key
1:19:31
partition key
1:19:31
partition key was chosen to be different for
1:19:33
was chosen to be different for
1:19:33
was chosen to be different for efficiency
1:19:35
efficiency
1:19:35
efficiency purposes and to ensure the balance
1:19:39
purposes and to ensure the balance
1:19:39
purposes and to ensure the balance in the data loads both for stored
1:19:42
in the data loads both for stored
1:19:42
in the data loads both for stored storage
1:19:43
storage
1:19:43
storage and retrieval if for some reason
1:19:46
and retrieval if for some reason
1:19:46
and retrieval if for some reason the choice of partition key had been
1:19:49
the choice of partition key had been
1:19:49
the choice of partition key had been different
1:19:49
different
1:19:50
different you may have uh you may be able to mix
1:19:53
you may have uh you may be able to mix
1:19:53
you may have uh you may be able to mix up
1:19:54
up
1:19:54
up physical documents uh into different
1:19:58
physical documents uh into different
1:19:58
physical documents uh into different logical containers but so
1:20:02
logical containers but so
1:20:02
logical containers but so you you have all kinds of flexible
1:20:05
you you have all kinds of flexible
1:20:06
you you have all kinds of flexible approaches that can be
1:20:07
approaches that can be
1:20:07
approaches that can be evaluated based on uh your use case and
1:20:11
evaluated based on uh your use case and
1:20:11
evaluated based on uh your use case and particularly
1:20:12
particularly
1:20:12
particularly based on your access patterns
1:20:17
based on your access patterns
1:20:17
based on your access patterns so now let's switch to a property graph
1:20:20
so now let's switch to a property graph
1:20:20
so now let's switch to a property graph example
1:20:21
example
1:20:21
example uh which was not covered in yesterday's
1:20:24
uh which was not covered in yesterday's
1:20:24
uh which was not covered in yesterday's session so here's an example of a
1:20:28
session so here's an example of a
1:20:28
session so here's an example of a property graph
1:20:30
property graph
1:20:30
property graph that would be used leveraged using um
1:20:34
that would be used leveraged using um
1:20:34
that would be used leveraged using um the gremlin api so uh
1:20:37
the gremlin api so uh
1:20:37
the gremlin api so uh for the example you know theoretical
1:20:40
for the example you know theoretical
1:20:40
for the example you know theoretical example we have
1:20:41
example we have
1:20:41
example we have a ring of fraudsters uh that
1:20:44
a ring of fraudsters uh that
1:20:44
a ring of fraudsters uh that share uh you know a subset
1:20:48
share uh you know a subset
1:20:48
share uh you know a subset of legitimate contact information that's
1:20:50
of legitimate contact information that's
1:20:50
of legitimate contact information that's been verified by
1:20:52
been verified by
1:20:52
been verified by financial institutions and like
1:20:55
financial institutions and like
1:20:55
financial institutions and like for example um phone numbers and
1:20:58
for example um phone numbers and
1:20:58
for example um phone numbers and addresses
1:21:00
addresses
1:21:00
addresses and the ring leaders
1:21:04
open accounts using these synths
1:21:07
open accounts using these synths
1:21:07
open accounts using these synths using synthetic scientific
1:21:10
using synthetic scientific
1:21:10
using synthetic scientific identities and create new accounts
1:21:14
identities and create new accounts
1:21:14
identities and create new accounts uh you know for credit cards and uh
1:21:17
uh you know for credit cards and uh
1:21:17
uh you know for credit cards and uh unsecured credit lines uh over draft
1:21:21
unsecured credit lines uh over draft
1:21:21
unsecured credit lines uh over draft protection
1:21:22
protection
1:21:22
protection uh etc etc so
1:21:25
uh etc etc so
1:21:25
uh etc etc so if we model this using
1:21:29
if we model this using
1:21:29
if we model this using accolade for example for the purpose
1:21:33
accolade for example for the purpose
1:21:33
accolade for example for the purpose of creating a structure to be used by
1:21:37
of creating a structure to be used by
1:21:37
of creating a structure to be used by the gremlin api we're going to create
1:21:40
the gremlin api we're going to create
1:21:40
the gremlin api we're going to create a node of vertex these circles
1:21:44
a node of vertex these circles
1:21:44
a node of vertex these circles here in the diagram for each concept in
1:21:47
here in the diagram for each concept in
1:21:47
here in the diagram for each concept in the example
1:21:49
the example
1:21:49
the example then we create relationships
1:21:52
then we create relationships
1:21:52
then we create relationships which are sometimes called edges
1:21:55
which are sometimes called edges
1:21:55
which are sometimes called edges between the nodes both the nodes
1:21:59
between the nodes both the nodes
1:21:59
between the nodes both the nodes and the edges can be enriched with
1:22:01
and the edges can be enriched with
1:22:01
and the edges can be enriched with properties so that's a
1:22:03
properties so that's a
1:22:03
properties so that's a special feature of property graph
1:22:05
special feature of property graph
1:22:05
special feature of property graph databases
1:22:07
databases
1:22:07
databases so the relationships and the nodes have
1:22:11
so the relationships and the nodes have
1:22:11
so the relationships and the nodes have their own properties and this allows you
1:22:14
their own properties and this allows you
1:22:14
their own properties and this allows you to create queries
1:22:15
to create queries
1:22:16
to create queries that traverse the graph to reveal
1:22:19
that traverse the graph to reveal
1:22:19
that traverse the graph to reveal connections
1:22:20
connections
1:22:20
connections that you didn't know existed and that's
1:22:23
that you didn't know existed and that's
1:22:24
that you didn't know existed and that's what uh you know for example we will
1:22:27
what uh you know for example we will
1:22:27
what uh you know for example we will reveal that
1:22:27
reveal that
1:22:28
reveal that phone numbers you know presumably
1:22:31
phone numbers you know presumably
1:22:31
phone numbers you know presumably legitimate phone numbers and addresses
1:22:33
legitimate phone numbers and addresses
1:22:33
legitimate phone numbers and addresses or bank accounts
1:22:35
or bank accounts
1:22:35
or bank accounts have been shared by different people and
1:22:37
have been shared by different people and
1:22:37
have been shared by different people and that's
1:22:38
that's
1:22:38
that's what would that's what would um reveal
1:22:41
what would that's what would um reveal
1:22:41
what would that's what would um reveal the fact
1:22:42
the fact
1:22:42
the fact that we should suspect a fraud in this
1:22:45
that we should suspect a fraud in this
1:22:45
that we should suspect a fraud in this instance
1:22:46
instance
1:22:46
instance and um you could forward engineer the
1:22:49
and um you could forward engineer the
1:22:49
and um you could forward engineer the structure to the cosmos db instance
1:22:52
structure to the cosmos db instance
1:22:52
structure to the cosmos db instance and then start loading your graph
1:22:56
and then start loading your graph
1:22:56
and then start loading your graph in cosmos db and start doing your
1:23:00
in cosmos db and start doing your
1:23:00
in cosmos db and start doing your queries that will
1:23:01
queries that will
1:23:01
queries that will reveal these relationships you didn't
1:23:04
reveal these relationships you didn't
1:23:04
reveal these relationships you didn't know existed
1:23:06
know existed
1:23:06
know existed okay so we're almost out of time
1:23:09
okay so we're almost out of time
1:23:09
okay so we're almost out of time as you can see while no sequel
1:23:13
as you can see while no sequel
1:23:13
as you can see while no sequel seems easy at first it actually requires
1:23:17
seems easy at first it actually requires
1:23:17
seems easy at first it actually requires a deeper skills to truly leverage the
1:23:20
a deeper skills to truly leverage the
1:23:20
a deeper skills to truly leverage the capabilities of the technology
1:23:22
capabilities of the technology
1:23:22
capabilities of the technology this translates into high performance
1:23:25
this translates into high performance
1:23:25
this translates into high performance here
1:23:25
here
1:23:25
here for your application greater developer
1:23:29
for your application greater developer
1:23:29
for your application greater developer productivity and lower cost of ownership
1:23:33
productivity and lower cost of ownership
1:23:33
productivity and lower cost of ownership so just to repeat myself a little bit
1:23:36
so just to repeat myself a little bit
1:23:36
so just to repeat myself a little bit um you know storing data correctly
1:23:39
um you know storing data correctly
1:23:40
um you know storing data correctly in cosmos db and leveraging um its
1:23:43
in cosmos db and leveraging um its
1:23:43
in cosmos db and leveraging um its uh capabilities will achieve the
1:23:47
uh capabilities will achieve the
1:23:47
uh capabilities will achieve the to achieve these uh objectives we just
1:23:50
to achieve these uh objectives we just
1:23:50
to achieve these uh objectives we just saw
1:23:50
saw
1:23:50
saw requires a complete mind shift you need
1:23:53
requires a complete mind shift you need
1:23:54
requires a complete mind shift you need to
1:23:54
to
1:23:54
to unlearn what you've known all along with
1:23:57
unlearn what you've known all along with
1:23:57
unlearn what you've known all along with relational databases
1:23:59
relational databases
1:23:59
relational databases and embrace this new approach
1:24:02
and embrace this new approach
1:24:02
and embrace this new approach of analyzing your access patterns
1:24:06
of analyzing your access patterns
1:24:06
of analyzing your access patterns and queries before you can
1:24:10
and queries before you can
1:24:10
and queries before you can structure the storage of your
1:24:13
structure the storage of your
1:24:13
structure the storage of your information
1:24:14
information
1:24:14
information following a data model that will
1:24:17
following a data model that will
1:24:17
following a data model that will continue to evolve
1:24:18
continue to evolve
1:24:18
continue to evolve with your application as it uh as it
1:24:22
with your application as it uh as it
1:24:22
with your application as it uh as it grows
1:24:23
grows
1:24:23
grows um in feature set and that will dictate
1:24:26
um in feature set and that will dictate
1:24:26
um in feature set and that will dictate how
1:24:27
how
1:24:27
how data is stored in
1:24:30
data is stored in
1:24:30
data is stored in in cosmos db you should pick
1:24:33
in cosmos db you should pick
1:24:33
in cosmos db you should pick the right tool for each use case and
1:24:37
the right tool for each use case and
1:24:37
the right tool for each use case and of course you know cosmos db offers you
1:24:41
of course you know cosmos db offers you
1:24:41
of course you know cosmos db offers you that
1:24:41
that
1:24:41
that choice of the right tool for the right
1:24:45
choice of the right tool for the right
1:24:45
choice of the right tool for the right job and if it doesn't you still have
1:24:49
job and if it doesn't you still have
1:24:49
job and if it doesn't you still have the rest of the azure offering so
1:24:53
the rest of the azure offering so
1:24:54
the rest of the azure offering so thank you for your attention we can now
1:24:56
thank you for your attention we can now
1:24:56
thank you for your attention we can now go to
1:24:57
go to
1:24:57
go to q a uh back to you guys
1:25:01
q a uh back to you guys
1:25:01
q a uh back to you guys awesome thank you pascal that was really
1:25:04
awesome thank you pascal that was really
1:25:04
awesome thank you pascal that was really great i i really loved
1:25:06
great i i really loved
1:25:06
great i i really loved in particular your very simple kind of
1:25:08
in particular your very simple kind of
1:25:08
in particular your very simple kind of characterization of um
1:25:11
characterization of um
1:25:11
characterization of um the difference between the app dev
1:25:12
the difference between the app dev
1:25:12
the difference between the app dev journey for relational
1:25:15
journey for relational
1:25:15
journey for relational and nosql it being kind of you know um
1:25:18
and nosql it being kind of you know um
1:25:18
and nosql it being kind of you know um uh an app dev centric data model
1:25:21
uh an app dev centric data model
1:25:21
uh an app dev centric data model uh uh process in fact i wonder if with
1:25:24
uh uh process in fact i wonder if with
1:25:24
uh uh process in fact i wonder if with the producer's help we can
1:25:26
the producer's help we can
1:25:26
the producer's help we can maybe get that slide up again i really
1:25:28
maybe get that slide up again i really
1:25:28
maybe get that slide up again i really love that simple slide
1:25:29
love that simple slide
1:25:29
love that simple slide or maybe not yes or no
1:25:31
or maybe not yes or no
1:25:31
or maybe not yes or no [Music]
1:25:32
[Music]
1:25:32
[Music] yeah that that the last but one slide i
1:25:35
yeah that that the last but one slide i
1:25:35
yeah that that the last but one slide i think it was
1:25:36
think it was
1:25:36
think it was this one no this the one before that
1:25:39
this one no this the one before that
1:25:39
this one no this the one before that yeah this one awesome yeah so it's just
1:25:42
yeah this one awesome yeah so it's just
1:25:42
yeah this one awesome yeah so it's just really that really spells it out for me
1:25:44
really that really spells it out for me
1:25:44
really that really spells it out for me uh i guess my question is um it follows
1:25:48
uh i guess my question is um it follows
1:25:48
uh i guess my question is um it follows from this that they're gonna be
1:25:50
from this that they're gonna be
1:25:50
from this that they're gonna be um things that are things that emerge
1:25:53
um things that are things that emerge
1:25:53
um things that are things that emerge for nosql databases like cosmos db
1:25:56
for nosql databases like cosmos db
1:25:56
for nosql databases like cosmos db that are good practices uh that would
1:25:58
that are good practices uh that would
1:25:58
that are good practices uh that would have been
1:25:59
have been
1:25:59
have been or would be anti-patterns in relational
1:26:02
or would be anti-patterns in relational
1:26:02
or would be anti-patterns in relational databases and vice versa things that
1:26:04
databases and vice versa things that
1:26:04
databases and vice versa things that uh that are let's say anti-patents but
1:26:07
uh that are let's say anti-patents but
1:26:07
uh that are let's say anti-patents but would be perhaps a good practice
1:26:09
would be perhaps a good practice
1:26:09
would be perhaps a good practice in a relational database i i wondered if
1:26:11
in a relational database i i wondered if
1:26:11
in a relational database i i wondered if you wanted to not to put you on the spot
1:26:13
you wanted to not to put you on the spot
1:26:13
you wanted to not to put you on the spot if you wanted to call out maybe
1:26:14
if you wanted to call out maybe
1:26:14
if you wanted to call out maybe a couple of of those that you've seen in
1:26:16
a couple of of those that you've seen in
1:26:16
a couple of of those that you've seen in the wild where people have
1:26:18
the wild where people have
1:26:18
the wild where people have uh fallen into the trap of either not
1:26:20
uh fallen into the trap of either not
1:26:20
uh fallen into the trap of either not doing something that they should do
1:26:22
doing something that they should do
1:26:22
doing something that they should do because they thought that was an
1:26:23
because they thought that was an
1:26:23
because they thought that was an anti-pattern or doing something
1:26:26
anti-pattern or doing something
1:26:26
anti-pattern or doing something that they thought was good where they've
1:26:28
that they thought was good where they've
1:26:28
that they thought was good where they've come from but it turns out to get them
1:26:30
come from but it turns out to get them
1:26:30
come from but it turns out to get them into
1:26:31
into
1:26:31
into difficulty yeah so excellent question
1:26:34
difficulty yeah so excellent question
1:26:34
difficulty yeah so excellent question so you know the flexibility and the
1:26:36
so you know the flexibility and the
1:26:36
so you know the flexibility and the power of
1:26:37
power of
1:26:37
power of json is something
1:26:40
json is something
1:26:40
json is something both wonderful and dangerous at the same
1:26:44
both wonderful and dangerous at the same
1:26:44
both wonderful and dangerous at the same time
1:26:44
time
1:26:44
time so it's wonderful because um
1:26:47
so it's wonderful because um
1:26:47
so it's wonderful because um it describes the schema which serves the
1:26:51
it describes the schema which serves the
1:26:51
it describes the schema which serves the schema on read
1:26:53
schema on read
1:26:53
schema on read approach and it's also incredibly
1:26:56
approach and it's also incredibly
1:26:56
approach and it's also incredibly flexible
1:26:57
flexible
1:26:57
flexible it can be polymorphic as we call it so
1:27:02
it can be polymorphic as we call it so
1:27:02
it can be polymorphic as we call it so you could have
1:27:03
you could have
1:27:03
you could have one document or one version of the
1:27:06
one document or one version of the
1:27:06
one document or one version of the information
1:27:07
information
1:27:07
information which is in one form and then
1:27:11
which is in one form and then
1:27:11
which is in one form and then the schema evolves and you have the
1:27:13
the schema evolves and you have the
1:27:13
the schema evolves and you have the coexistence
1:27:15
coexistence
1:27:15
coexistence of another set of information for the
1:27:18
of another set of information for the
1:27:18
of another set of information for the same concept
1:27:19
same concept
1:27:19
same concept that follows a different form
1:27:23
that follows a different form
1:27:23
that follows a different form and this polymorphism is wonderful
1:27:27
and this polymorphism is wonderful
1:27:27
and this polymorphism is wonderful because it's flexible it helps you
1:27:29
because it's flexible it helps you
1:27:29
because it's flexible it helps you evolve very quickly but it can be
1:27:32
evolve very quickly but it can be
1:27:32
evolve very quickly but it can be dangerous
1:27:33
dangerous
1:27:33
dangerous if you don't do it with rigor
1:27:36
if you don't do it with rigor
1:27:36
if you don't do it with rigor it can lead to chaos if you're not
1:27:39
it can lead to chaos if you're not
1:27:39
it can lead to chaos if you're not organized
1:27:40
organized
1:27:40
organized and so many people think in the
1:27:43
and so many people think in the
1:27:43
and so many people think in the beginning
1:27:44
beginning
1:27:44
beginning that you can let that grow
1:27:48
that you can let that grow
1:27:48
that you can let that grow organically and then they realize that
1:27:51
organically and then they realize that
1:27:51
organically and then they realize that maybe you know there are other
1:27:53
maybe you know there are other
1:27:53
maybe you know there are other applications accessing the same
1:27:55
applications accessing the same
1:27:55
applications accessing the same information
1:27:57
information
1:27:57
information and that means that each of the
1:27:59
and that means that each of the
1:27:59
and that means that each of the applications
1:28:01
applications
1:28:01
applications need to be enhanced to understand the
1:28:05
need to be enhanced to understand the
1:28:05
need to be enhanced to understand the different forms that the data might
1:28:07
different forms that the data might
1:28:07
different forms that the data might take in your document store and so
1:28:10
take in your document store and so
1:28:10
take in your document store and so um data migration
1:28:15
um data migration
1:28:15
um data migration may be maybe best advised
1:28:18
may be maybe best advised
1:28:18
may be maybe best advised to be performed because you don't want
1:28:21
to be performed because you don't want
1:28:21
to be performed because you don't want to let this grow out of
1:28:23
to let this grow out of
1:28:23
to let this grow out of hand the advantage is that you can do
1:28:26
hand the advantage is that you can do
1:28:26
hand the advantage is that you can do this data migration
1:28:28
this data migration
1:28:28
this data migration you know at your leisure whenever needed
1:28:32
you know at your leisure whenever needed
1:28:32
you know at your leisure whenever needed instead of being forced to do it when
1:28:34
instead of being forced to do it when
1:28:34
instead of being forced to do it when you do a schema change like you would
1:28:37
you do a schema change like you would
1:28:37
you do a schema change like you would in a relational database so that's one
1:28:39
in a relational database so that's one
1:28:39
in a relational database so that's one example of
1:28:41
example of
1:28:41
example of dangers that might come up uh with
1:28:45
dangers that might come up uh with
1:28:45
dangers that might come up uh with this flexibility the other one i
1:28:47
this flexibility the other one i
1:28:47
this flexibility the other one i mentioned already in one of the slides
1:28:49
mentioned already in one of the slides
1:28:49
mentioned already in one of the slides is the risk of this
1:28:53
is the risk of this
1:28:53
is the risk of this these documents becoming dumping grounds
1:28:56
these documents becoming dumping grounds
1:28:56
these documents becoming dumping grounds i have seen
1:28:58
i have seen
1:28:58
i have seen ridiculously big documents you know
1:29:01
ridiculously big documents you know
1:29:01
ridiculously big documents you know documents that
1:29:02
documents that
1:29:02
documents that may be megabytes uh
1:29:05
may be megabytes uh
1:29:05
may be megabytes uh of information and given you know the
1:29:08
of information and given you know the
1:29:08
of information and given you know the the performance the costing model etc
1:29:11
the performance the costing model etc
1:29:11
the performance the costing model etc etc
1:29:12
etc
1:29:12
etc you know this is not advised um you know
1:29:15
you know this is not advised um you know
1:29:15
you know this is not advised um you know it should remain pragmatic
1:29:17
it should remain pragmatic
1:29:18
it should remain pragmatic and the documents should not grow out of
1:29:20
and the documents should not grow out of
1:29:20
and the documents should not grow out of hand so i would
1:29:21
hand so i would
1:29:21
hand so i would want people against these traps
1:29:24
want people against these traps
1:29:24
want people against these traps that are really allowed by the
1:29:27
that are really allowed by the
1:29:27
that are really allowed by the flexibility of json
1:29:29
flexibility of json
1:29:29
flexibility of json and by the fact that you don't have the
1:29:31
and by the fact that you don't have the
1:29:31
and by the fact that you don't have the rules of normalizations that are sort of
1:29:34
rules of normalizations that are sort of
1:29:34
rules of normalizations that are sort of guard rails to help you so be careful
1:29:37
guard rails to help you so be careful
1:29:37
guard rails to help you so be careful about
1:29:38
about
1:29:38
about you know this flexibility getting out of
1:29:40
you know this flexibility getting out of
1:29:40
you know this flexibility getting out of hand
1:29:42
hand
1:29:42
hand it's a great answer with great
1:29:44
it's a great answer with great
1:29:44
it's a great answer with great responsibility comes uh
1:29:46
responsibility comes uh
1:29:46
responsibility comes uh with great power comes great
1:29:47
with great power comes great
1:29:47
with great power comes great responsibility right exactly
1:29:52
responsibility right exactly
1:29:52
responsibility right exactly anyone else have questions um we did get
1:29:55
anyone else have questions um we did get
1:29:55
anyone else have questions um we did get a question from our viewers
1:29:57
a question from our viewers
1:29:57
a question from our viewers if you can quickly explain um uh the
1:30:00
if you can quickly explain um uh the
1:30:00
if you can quickly explain um uh the schema in the context of transactional
1:30:02
schema in the context of transactional
1:30:02
schema in the context of transactional store
1:30:04
store
1:30:04
store yeah so um obviously transactions are
1:30:08
yeah so um obviously transactions are
1:30:08
yeah so um obviously transactions are well served by uh relational databases
1:30:12
well served by uh relational databases
1:30:12
well served by uh relational databases uh when it comes to nosql databases
1:30:15
uh when it comes to nosql databases
1:30:15
uh when it comes to nosql databases these transactions uh need to be
1:30:18
these transactions uh need to be
1:30:18
these transactions uh need to be built in uh the application um
1:30:22
built in uh the application um
1:30:22
built in uh the application um because the engine does not uh enforce
1:30:25
because the engine does not uh enforce
1:30:25
because the engine does not uh enforce the transaction
1:30:26
the transaction
1:30:26
the transaction um you know this comes to the concept of
1:30:30
um you know this comes to the concept of
1:30:30
um you know this comes to the concept of uh also eventual consistency which i did
1:30:33
uh also eventual consistency which i did
1:30:33
uh also eventual consistency which i did not talk about
1:30:34
not talk about
1:30:34
not talk about which some of the old school
1:30:37
which some of the old school
1:30:37
which some of the old school people you know have a problem with that
1:30:41
people you know have a problem with that
1:30:41
people you know have a problem with that nosql databases of eventual consistency
1:30:45
nosql databases of eventual consistency
1:30:45
nosql databases of eventual consistency i think in these terms that eventual
1:30:48
i think in these terms that eventual
1:30:48
i think in these terms that eventual cons
1:30:49
cons
1:30:49
cons that perfect transactional consistency
1:30:53
that perfect transactional consistency
1:30:53
that perfect transactional consistency is only possible when you
1:30:56
is only possible when you
1:30:56
is only possible when you own the full chain of the transaction
1:31:01
own the full chain of the transaction
1:31:01
own the full chain of the transaction which is easy within your own database
1:31:04
which is easy within your own database
1:31:04
which is easy within your own database but
1:31:04
but
1:31:04
but when you're transferring money from one
1:31:07
when you're transferring money from one
1:31:07
when you're transferring money from one bank to another bank it doesn't happen
1:31:09
bank to another bank it doesn't happen
1:31:09
bank to another bank it doesn't happen so get used to the fact that eventual
1:31:12
so get used to the fact that eventual
1:31:12
so get used to the fact that eventual consistency
1:31:13
consistency
1:31:13
consistency is just a very efficient way of doing
1:31:17
is just a very efficient way of doing
1:31:17
is just a very efficient way of doing things and
1:31:17
things and
1:31:17
things and building and you should build in your
1:31:20
building and you should build in your
1:31:20
building and you should build in your application
1:31:21
application
1:31:21
application the checks and balances to make sure
1:31:24
the checks and balances to make sure
1:31:24
the checks and balances to make sure that
1:31:25
that
1:31:25
that nothing goes up in smoke
1:31:31
nothing goes up in smoke
1:31:31
nothing goes up in smoke all right thank you so much pascal um up
1:31:34
all right thank you so much pascal um up
1:31:34
all right thank you so much pascal um up next
1:31:34
next
1:31:34
next we have a quick video on throughput and
1:31:37
we have a quick video on throughput and
1:31:37
we have a quick video on throughput and pricing
1:31:38
pricing
1:31:38
pricing uh so for a long time cosmo cv had only
1:31:41
uh so for a long time cosmo cv had only
1:31:41
uh so for a long time cosmo cv had only one way of provisioning throughput
1:31:42
one way of provisioning throughput
1:31:42
one way of provisioning throughput um and then customers would have a would
1:31:44
um and then customers would have a would
1:31:44
um and then customers would have a would have to really know their workload very
1:31:46
have to really know their workload very
1:31:46
have to really know their workload very well
1:31:47
well
1:31:47
well um if they wanted to effectively and
1:31:48
um if they wanted to effectively and
1:31:48
um if they wanted to effectively and officially provision that throughput
1:31:50
officially provision that throughput
1:31:50
officially provision that throughput um and so we realized that customers are
1:31:52
um and so we realized that customers are
1:31:52
um and so we realized that customers are different and we all have different
1:31:54
different and we all have different
1:31:54
different and we all have different needs so
1:31:55
needs so
1:31:55
needs so we've been working hard to create new
1:31:56
we've been working hard to create new
1:31:56
we've been working hard to create new ways for customers to provision
1:31:58
ways for customers to provision
1:31:58
ways for customers to provision on throughput including with auto scaled
1:32:00
on throughput including with auto scaled
1:32:00
on throughput including with auto scaled and serverless so watch this a quick
1:32:02
and serverless so watch this a quick
1:32:02
and serverless so watch this a quick video to explain differences between the
1:32:04
video to explain differences between the
1:32:04
video to explain differences between the two
1:32:07
azure cosmos db helps you get more value
1:32:09
azure cosmos db helps you get more value
1:32:09
azure cosmos db helps you get more value for your money by making it easy to
1:32:11
for your money by making it easy to
1:32:11
for your money by making it easy to manage
1:32:11
manage
1:32:12
manage the components you pay for database
1:32:14
the components you pay for database
1:32:14
the components you pay for database operations and storage
1:32:16
operations and storage
1:32:16
operations and storage the cost to perform database operations
1:32:18
the cost to perform database operations
1:32:18
the cost to perform database operations including memory
1:32:19
including memory
1:32:19
including memory cpu and iops is normalized and expressed
1:32:22
cpu and iops is normalized and expressed
1:32:22
cpu and iops is normalized and expressed as a request unit
1:32:24
as a request unit
1:32:24
as a request unit more request units are charged for more
1:32:26
more request units are charged for more
1:32:26
more request units are charged for more demanding activities
1:32:28
demanding activities
1:32:28
demanding activities for database operations you can select
1:32:30
for database operations you can select
1:32:30
for database operations you can select one of two models
1:32:31
one of two models
1:32:31
one of two models provisioned throughput for serverless
1:32:33
provisioned throughput for serverless
1:32:33
provisioned throughput for serverless consumption
1:32:35
consumption
1:32:35
consumption provisioned throughput is the capacity
1:32:37
provisioned throughput is the capacity
1:32:37
provisioned throughput is the capacity you allocate for database operations
1:32:39
you allocate for database operations
1:32:39
you allocate for database operations measured in the number of request units
1:32:41
measured in the number of request units
1:32:41
measured in the number of request units per second and billed hourly
1:32:43
per second and billed hourly
1:32:43
per second and billed hourly it works best for workloads that always
1:32:45
it works best for workloads that always
1:32:45
it works best for workloads that always have some traffic and require high
1:32:47
have some traffic and require high
1:32:47
have some traffic and require high performance
1:32:48
performance
1:32:48
performance slas if the traffic is predictable you
1:32:50
slas if the traffic is predictable you
1:32:50
slas if the traffic is predictable you can use standard provision throughput
1:32:52
can use standard provision throughput
1:32:52
can use standard provision throughput to manually set and adjust capacity as
1:32:55
to manually set and adjust capacity as
1:32:55
to manually set and adjust capacity as needed
1:32:56
needed
1:32:56
needed if the traffic is unpredictable you can
1:32:58
if the traffic is unpredictable you can
1:32:58
if the traffic is unpredictable you can use auto scale provision throughput
1:33:00
use auto scale provision throughput
1:33:00
use auto scale provision throughput to instantaneously and automatically
1:33:02
to instantaneously and automatically
1:33:02
to instantaneously and automatically adjust capacity
1:33:03
adjust capacity
1:33:03
adjust capacity between 10 and 100 of your set limit
1:33:07
between 10 and 100 of your set limit
1:33:07
between 10 and 100 of your set limit auto scale becomes more cost effective
1:33:09
auto scale becomes more cost effective
1:33:09
auto scale becomes more cost effective than standard when traffic is
1:33:11
than standard when traffic is
1:33:11
than standard when traffic is unpredictable and not close to maximum
1:33:13
unpredictable and not close to maximum
1:33:13
unpredictable and not close to maximum capacity most of the time
1:33:16
capacity most of the time
1:33:16
capacity most of the time provision throughput may not suit
1:33:17
provision throughput may not suit
1:33:17
provision throughput may not suit workloads with only occasional database
1:33:19
workloads with only occasional database
1:33:19
workloads with only occasional database operations and lower performance
1:33:21
operations and lower performance
1:33:21
operations and lower performance requirements
1:33:22
requirements
1:33:22
requirements these applications can benefit from the
1:33:24
these applications can benefit from the
1:33:24
these applications can benefit from the serverless model
1:33:26
serverless model
1:33:26
serverless model while it has a higher unit cost it's
1:33:28
while it has a higher unit cost it's
1:33:28
while it has a higher unit cost it's consumption based and only charges for
1:33:30
consumption based and only charges for
1:33:30
consumption based and only charges for the request units used for database
1:33:32
the request units used for database
1:33:32
the request units used for database operation
1:33:34
operation
1:33:34
operation with consumed storage fees are charged
1:33:36
with consumed storage fees are charged
1:33:36
with consumed storage fees are charged for the total gigabytes used per month
1:33:38
for the total gigabytes used per month
1:33:38
for the total gigabytes used per month for both transactional
1:33:39
for both transactional
1:33:39
for both transactional and analytical storage you also pay for
1:33:42
and analytical storage you also pay for
1:33:42
and analytical storage you also pay for storage i o
1:33:43
storage i o
1:33:43
storage i o and analytical storage get the most
1:33:46
and analytical storage get the most
1:33:46
and analytical storage get the most value from your workloads by
1:33:48
value from your workloads by
1:33:48
value from your workloads by understanding the components you're
1:33:49
understanding the components you're
1:33:49
understanding the components you're built for in azure cosmos db
1:33:56
[Music]
1:33:59
[Music]
1:33:59
[Music] all right so these are definitely great
1:34:01
all right so these are definitely great
1:34:01
all right so these are definitely great offerings that we have
1:34:02
offerings that we have
1:34:02
offerings that we have make sure to definitely take a look so
1:34:04
make sure to definitely take a look so
1:34:04
make sure to definitely take a look so up next we have staff
1:34:06
up next we have staff
1:34:06
up next we have staff steffian wickers he's an a cloud
1:34:09
steffian wickers he's an a cloud
1:34:09
steffian wickers he's an a cloud architect
1:34:10
architect
1:34:10
architect and he is speaking on cloud solutions
1:34:12
and he is speaking on cloud solutions
1:34:12
and he is speaking on cloud solutions with cosmos tv
1:34:15
with cosmos tv
1:34:15
with cosmos tv hi everyone i hope you're enjoying the
1:34:17
hi everyone i hope you're enjoying the
1:34:17
hi everyone i hope you're enjoying the azure cosmos db conf
1:34:19
azure cosmos db conf
1:34:19
azure cosmos db conf for now and there were some great
1:34:21
for now and there were some great
1:34:21
for now and there were some great sessions um i will continue
1:34:23
sessions um i will continue
1:34:23
sessions um i will continue with my story on integrating azure
1:34:25
with my story on integrating azure
1:34:25
with my story on integrating azure cosmos db in your cloud solutions so
1:34:28
cosmos db in your cloud solutions so
1:34:28
cosmos db in your cloud solutions so i'm stephen riggers technical
1:34:29
i'm stephen riggers technical
1:34:29
i'm stephen riggers technical integration architect at hso
1:34:32
integration architect at hso
1:34:33
integration architect at hso a little bit of myself i'm an azure mvp
1:34:35
a little bit of myself i'm an azure mvp
1:34:35
a little bit of myself i'm an azure mvp i started out in the integration
1:34:37
i started out in the integration
1:34:37
i started out in the integration space and since 2015 and now in the
1:34:39
space and since 2015 and now in the
1:34:39
space and since 2015 and now in the azure skew
1:34:40
azure skew
1:34:40
azure skew with the program i write trading for
1:34:42
with the program i write trading for
1:34:42
with the program i write trading for code so this is a
1:34:44
code so this is a
1:34:44
code so this is a an international website about all kinds
1:34:46
an international website about all kinds
1:34:46
an international website about all kinds of technologies including cloud
1:34:47
of technologies including cloud
1:34:47
of technologies including cloud technologies
1:34:48
technologies
1:34:48
technologies like um aws azure
1:34:52
like um aws azure
1:34:52
like um aws azure uh google cloud platform also organizer
1:34:54
uh google cloud platform also organizer
1:34:54
uh google cloud platform also organizer some of the events that are listed here
1:34:56
some of the events that are listed here
1:34:56
some of the events that are listed here user groups where sometimes also cosmos
1:34:59
user groups where sometimes also cosmos
1:34:59
user groups where sometimes also cosmos db is one of the topics
1:35:00
db is one of the topics
1:35:00
db is one of the topics and besides this i like running so
1:35:02
and besides this i like running so
1:35:02
and besides this i like running so hopefully when the global
1:35:04
hopefully when the global
1:35:04
hopefully when the global pandemic will go away i'm able to
1:35:06
pandemic will go away i'm able to
1:35:06
pandemic will go away i'm able to continue my marathon running
1:35:09
continue my marathon running
1:35:09
continue my marathon running okay so what i'm going to do is kind of
1:35:12
okay so what i'm going to do is kind of
1:35:12
okay so what i'm going to do is kind of tell my cosmos db story from my view
1:35:15
tell my cosmos db story from my view
1:35:15
tell my cosmos db story from my view with some experience i had with this
1:35:18
with some experience i had with this
1:35:18
with some experience i had with this service or this capability in azure um
1:35:22
service or this capability in azure um
1:35:22
service or this capability in azure um also you know some use cases where you
1:35:23
also you know some use cases where you
1:35:24
also you know some use cases where you know i was integrating azure cosmos db
1:35:26
know i was integrating azure cosmos db
1:35:26
know i was integrating azure cosmos db research
1:35:27
research
1:35:27
research leveraging your change feed capability
1:35:29
leveraging your change feed capability
1:35:29
leveraging your change feed capability and also some integration with
1:35:30
and also some integration with
1:35:30
and also some integration with other azure services
1:35:34
other azure services
1:35:34
other azure services so initially when cosmos db came to play
1:35:37
so initially when cosmos db came to play
1:35:37
so initially when cosmos db came to play so this is a little bit i also like to
1:35:38
so this is a little bit i also like to
1:35:38
so this is a little bit i also like to to learn a bit about the background of
1:35:40
to learn a bit about the background of
1:35:40
to learn a bit about the background of the service
1:35:40
the service
1:35:40
the service this is a couple years ago um nema remy
1:35:43
this is a couple years ago um nema remy
1:35:43
this is a couple years ago um nema remy and just
1:35:44
and just
1:35:44
and just sql pass introduced the fact that hey
1:35:46
sql pass introduced the fact that hey
1:35:46
sql pass introduced the fact that hey there's a need because the global
1:35:49
there's a need because the global
1:35:49
there's a need because the global increase of data that you know kind of
1:35:50
increase of data that you know kind of
1:35:50
increase of data that you know kind of need what's called an earthly database
1:35:52
need what's called an earthly database
1:35:52
need what's called an earthly database kind of get your scalability consistency
1:35:54
kind of get your scalability consistency
1:35:54
kind of get your scalability consistency ever in the world so kind of a project
1:35:57
ever in the world so kind of a project
1:35:57
ever in the world so kind of a project started with what's called
1:35:58
started with what's called
1:35:58
started with what's called project florence by dharmashukla which
1:36:00
project florence by dharmashukla which
1:36:00
project florence by dharmashukla which kind of you know
1:36:01
kind of you know
1:36:01
kind of you know something with these trends and creating
1:36:03
something with these trends and creating
1:36:03
something with these trends and creating a database it became azure cosmos db
1:36:06
a database it became azure cosmos db
1:36:06
a database it became azure cosmos db um around the time in 2017 the service
1:36:09
um around the time in 2017 the service
1:36:09
um around the time in 2017 the service was being announced
1:36:10
was being announced
1:36:10
was being announced and i was trying to get a little bit
1:36:11
and i was trying to get a little bit
1:36:11
and i was trying to get a little bit more familiar with the service i was
1:36:13
more familiar with the service i was
1:36:13
more familiar with the service i was working for a customer that was looking
1:36:15
working for a customer that was looking
1:36:15
working for a customer that was looking for instances into the graph model and
1:36:16
for instances into the graph model and
1:36:16
for instances into the graph model and i think this has already been explained
1:36:18
i think this has already been explained
1:36:18
i think this has already been explained in previous sessions so you have like
1:36:20
in previous sessions so you have like
1:36:20
in previous sessions so you have like your
1:36:20
your
1:36:20
your notes or vertices and you have
1:36:24
notes or vertices and you have
1:36:24
notes or vertices and you have edges so the combination with like
1:36:26
edges so the combination with like
1:36:26
edges so the combination with like someone knows someone else um
1:36:27
someone knows someone else um
1:36:27
someone knows someone else um someone uses certain devices you can see
1:36:29
someone uses certain devices you can see
1:36:29
someone uses certain devices you can see depicted in this picture
1:36:31
depicted in this picture
1:36:31
depicted in this picture it's the underpinning kind of model you
1:36:33
it's the underpinning kind of model you
1:36:33
it's the underpinning kind of model you also find in
1:36:35
also find in
1:36:35
also find in linkedin and facebook so i was
1:36:38
linkedin and facebook so i was
1:36:38
linkedin and facebook so i was getting in familiarize because of a
1:36:40
getting in familiarize because of a
1:36:40
getting in familiarize because of a project i was working for so this was
1:36:42
project i was working for so this was
1:36:42
project i was working for so this was kind of a knowledge base for
1:36:44
kind of a knowledge base for
1:36:44
kind of a knowledge base for um tax returns so this was done by
1:36:46
um tax returns so this was done by
1:36:46
um tax returns so this was done by rebusiness part of the relics group so
1:36:48
rebusiness part of the relics group so
1:36:48
rebusiness part of the relics group so tax returns for the
1:36:50
tax returns for the
1:36:50
tax returns for the dutch tax and they already had this old
1:36:53
dutch tax and they already had this old
1:36:53
dutch tax and they already had this old knowledge-based implementation and it
1:36:55
knowledge-based implementation and it
1:36:55
knowledge-based implementation and it was an is solution on
1:36:57
was an is solution on
1:36:57
was an is solution on combination with solar technology that's
1:36:59
combination with solar technology that's
1:36:59
combination with solar technology that's kind of a search engine
1:37:00
kind of a search engine
1:37:00
kind of a search engine and the company wanted to move forward
1:37:02
and the company wanted to move forward
1:37:02
and the company wanted to move forward they wanted to embrace the cloud they
1:37:03
they wanted to embrace the cloud they
1:37:03
they wanted to embrace the cloud they want to kind of
1:37:04
want to kind of
1:37:04
want to kind of use a cloud platform like azure with the
1:37:06
use a cloud platform like azure with the
1:37:06
use a cloud platform like azure with the ph go servers but also completely go
1:37:08
ph go servers but also completely go
1:37:08
ph go servers but also completely go platform as a service also to create
1:37:11
platform as a service also to create
1:37:11
platform as a service also to create this future-proof knowledge base to give
1:37:13
this future-proof knowledge base to give
1:37:13
this future-proof knowledge base to give a little bit more richer search
1:37:14
a little bit more richer search
1:37:14
a little bit more richer search experience for the customers
1:37:16
experience for the customers
1:37:16
experience for the customers because of the tax returns it's kind of
1:37:18
because of the tax returns it's kind of
1:37:18
because of the tax returns it's kind of a subscription model it's kind of what's
1:37:19
a subscription model it's kind of what's
1:37:19
a subscription model it's kind of what's called
1:37:20
called
1:37:20
called almanac enabling people to fill in
1:37:23
almanac enabling people to fill in
1:37:23
almanac enabling people to fill in or small or medium businesses to fill in
1:37:26
or small or medium businesses to fill in
1:37:26
or small or medium businesses to fill in their tax returns in a way that you know
1:37:28
their tax returns in a way that you know
1:37:28
their tax returns in a way that you know grossest that much money so the
1:37:30
grossest that much money so the
1:37:30
grossest that much money so the functional components is on one end you
1:37:32
functional components is on one end you
1:37:32
functional components is on one end you have the
1:37:32
have the
1:37:32
have the content creation which is a cms system
1:37:35
content creation which is a cms system
1:37:35
content creation which is a cms system that flows in with
1:37:36
that flows in with
1:37:36
that flows in with jurisprudence text laws etc some leading
1:37:39
jurisprudence text laws etc some leading
1:37:39
jurisprudence text laws etc some leading comments explaining how
1:37:42
comments explaining how
1:37:42
comments explaining how you should interpret those tax laws
1:37:44
you should interpret those tax laws
1:37:44
you should interpret those tax laws which counts
1:37:45
which counts
1:37:45
which counts then goes through what's called content
1:37:47
then goes through what's called content
1:37:47
then goes through what's called content constitution so it's ingested
1:37:49
constitution so it's ingested
1:37:49
constitution so it's ingested it's cleansed it's subsequently pushed
1:37:51
it's cleansed it's subsequently pushed
1:37:51
it's cleansed it's subsequently pushed into
1:37:52
into
1:37:52
into a knowledge base where you have your
1:37:55
a knowledge base where you have your
1:37:55
a knowledge base where you have your content types and relations between them
1:37:57
content types and relations between them
1:37:57
content types and relations between them and index and all that stuff so that's
1:37:59
and index and all that stuff so that's
1:37:59
and index and all that stuff so that's your knowledge base so this is kind of
1:38:01
your knowledge base so this is kind of
1:38:01
your knowledge base so this is kind of the functional aspect
1:38:02
the functional aspect
1:38:02
the functional aspect if we look at the the building block so
1:38:04
if we look at the the building block so
1:38:04
if we look at the the building block so we kind of set up this this proof of
1:38:05
we kind of set up this this proof of
1:38:05
we kind of set up this this proof of architecture where we embrace
1:38:07
architecture where we embrace
1:38:07
architecture where we embrace some of the azure components including
1:38:09
some of the azure components including
1:38:09
some of the azure components including cosmos db
1:38:10
cosmos db
1:38:10
cosmos db and some of the parts like integrate
1:38:12
and some of the parts like integrate
1:38:12
and some of the parts like integrate matching merge was done through
1:38:13
matching merge was done through
1:38:13
matching merge was done through net or web jobs and functions to push
1:38:17
net or web jobs and functions to push
1:38:17
net or web jobs and functions to push one end the the content you know text
1:38:19
one end the the content you know text
1:38:19
one end the the content you know text laws during sprinkles
1:38:21
laws during sprinkles
1:38:21
laws during sprinkles all these kind of content and content
1:38:23
all these kind of content and content
1:38:23
all these kind of content and content types into
1:38:24
types into
1:38:24
types into a document db store which was
1:38:26
a document db store which was
1:38:26
a document db store which was subsequently indexed leveraging the
1:38:28
subsequently indexed leveraging the
1:38:28
subsequently indexed leveraging the integration with search and on the other
1:38:30
integration with search and on the other
1:38:30
integration with search and on the other end
1:38:31
end
1:38:31
end another part was being used or at least
1:38:34
another part was being used or at least
1:38:34
another part was being used or at least the
1:38:34
the
1:38:34
the you had the the document model on one
1:38:36
you had the the document model on one
1:38:36
you had the the document model on one end and you had the graph graph model on
1:38:38
end and you had the graph graph model on
1:38:38
end and you had the graph graph model on the other end
1:38:39
the other end
1:38:39
the other end to set the relationships between these
1:38:40
to set the relationships between these
1:38:40
to set the relationships between these content types also
1:38:42
content types also
1:38:42
content types also to for instance implement the type of
1:38:45
to for instance implement the type of
1:38:45
to for instance implement the type of hierarchy
1:38:46
hierarchy
1:38:46
hierarchy and maybe also via some recommendation
1:38:47
and maybe also via some recommendation
1:38:47
and maybe also via some recommendation with regards to finding related content
1:38:50
with regards to finding related content
1:38:50
with regards to finding related content to the text
1:38:51
to the text
1:38:51
to the text except the text laws itself and then
1:38:53
except the text laws itself and then
1:38:53
except the text laws itself and then these were exposed through
1:38:55
these were exposed through
1:38:55
these were exposed through content api which was net apis
1:38:59
content api which was net apis
1:38:59
content api which was net apis functions or other type of code
1:39:02
functions or other type of code
1:39:02
functions or other type of code exposed through web apps
1:39:06
exposed through web apps
1:39:06
exposed through web apps so kind of the solution architecture in
1:39:08
so kind of the solution architecture in
1:39:08
so kind of the solution architecture in general is you have you see a cms a
1:39:10
general is you have you see a cms a
1:39:10
general is you have you see a cms a content management system output which
1:39:11
content management system output which
1:39:12
content management system output which is the text login experience etc
1:39:14
is the text login experience etc
1:39:14
is the text login experience etc the content itself being pushed into a
1:39:16
the content itself being pushed into a
1:39:16
the content itself being pushed into a documentdb model as
1:39:18
documentdb model as
1:39:18
documentdb model as text with some characteristics and
1:39:19
text with some characteristics and
1:39:19
text with some characteristics and properties indexed
1:39:21
properties indexed
1:39:21
properties indexed and through endpoints enabled to a
1:39:23
and through endpoints enabled to a
1:39:24
and through endpoints enabled to a front-end
1:39:24
front-end
1:39:24
front-end similarly the meta data model around all
1:39:28
similarly the meta data model around all
1:39:28
similarly the meta data model around all these content types were pushed into a
1:39:30
these content types were pushed into a
1:39:30
these content types were pushed into a graph model
1:39:31
graph model
1:39:31
graph model and then also being exposed through an
1:39:33
and then also being exposed through an
1:39:33
and then also being exposed through an api and when
1:39:34
api and when
1:39:34
api and when the call was made to that graph model
1:39:36
the call was made to that graph model
1:39:36
the call was made to that graph model the kremlin api was being leveraged so
1:39:40
the kremlin api was being leveraged so
1:39:40
the kremlin api was being leveraged so going forward kind of what meant
1:39:41
going forward kind of what meant
1:39:41
going forward kind of what meant is that when someone looked up certain
1:39:43
is that when someone looked up certain
1:39:43
is that when someone looked up certain tax laws
1:39:45
tax laws
1:39:45
tax laws there was an initial call to the indexed
1:39:48
there was an initial call to the indexed
1:39:48
there was an initial call to the indexed endpoints that sits on top of the
1:39:49
endpoints that sits on top of the
1:39:49
endpoints that sits on top of the documentdb store but subsequently
1:39:51
documentdb store but subsequently
1:39:51
documentdb store but subsequently another call was being done
1:39:52
another call was being done
1:39:52
another call was being done to also get the related content
1:39:56
to also get the related content
1:39:56
to also get the related content provisioned or at least made available
1:39:59
provisioned or at least made available
1:39:59
provisioned or at least made available in
1:39:59
in
1:39:59
in the front end this way giving a more
1:40:01
the front end this way giving a more
1:40:01
the front end this way giving a more richer experience that
1:40:03
richer experience that
1:40:03
richer experience that briefly was not aware because of the
1:40:04
briefly was not aware because of the
1:40:04
briefly was not aware because of the relations between the content
1:40:06
relations between the content
1:40:06
relations between the content and then subsequently if you clicked on
1:40:08
and then subsequently if you clicked on
1:40:08
and then subsequently if you clicked on one of the related content you wanted to
1:40:10
one of the related content you wanted to
1:40:10
one of the related content you wanted to tell now maybe more in a leaning kind of
1:40:12
tell now maybe more in a leaning kind of
1:40:12
tell now maybe more in a leaning kind of comment on explanatory how to interpret
1:40:14
comment on explanatory how to interpret
1:40:14
comment on explanatory how to interpret the law
1:40:14
the law
1:40:14
the law then there was another call to the
1:40:15
then there was another call to the
1:40:16
then there was another call to the content itself
1:40:18
content itself
1:40:18
content itself and this kind of led to a reference case
1:40:20
and this kind of led to a reference case
1:40:20
and this kind of led to a reference case because we kind of also um
1:40:23
because we kind of also um
1:40:23
because we kind of also um as a supplier for the solution we're
1:40:24
as a supplier for the solution we're
1:40:24
as a supplier for the solution we're working closely together back then with
1:40:26
working closely together back then with
1:40:26
working closely together back then with microsoft with
1:40:27
microsoft with
1:40:27
microsoft with a former pm called danny lee who helped
1:40:31
a former pm called danny lee who helped
1:40:31
a former pm called danny lee who helped us out
1:40:31
us out
1:40:31
us out validating also our proof of
1:40:33
validating also our proof of
1:40:33
validating also our proof of architecture and see if this
1:40:37
architecture and see if this
1:40:37
architecture and see if this things we found out and these learnings
1:40:39
things we found out and these learnings
1:40:39
things we found out and these learnings to really push forward to an actual
1:40:41
to really push forward to an actual
1:40:41
to really push forward to an actual solution itself
1:40:42
solution itself
1:40:42
solution itself and this is kind of what you can find
1:40:43
and this is kind of what you can find
1:40:44
and this is kind of what you can find back on online as well
1:40:47
back on online as well
1:40:47
back on online as well so going forward so for my learnings of
1:40:49
so going forward so for my learnings of
1:40:49
so going forward so for my learnings of this project
1:40:50
this project
1:40:50
this project i proceed on to another project i'm you
1:40:52
i proceed on to another project i'm you
1:40:52
i proceed on to another project i'm you know usually doing integrations between
1:40:55
know usually doing integrations between
1:40:55
know usually doing integrations between systems on-prem in the cloud or doing
1:40:57
systems on-prem in the cloud or doing
1:40:58
systems on-prem in the cloud or doing complete cloud of native integra
1:40:59
complete cloud of native integra
1:40:59
complete cloud of native integra cloud native integrations um but for
1:41:02
cloud native integrations um but for
1:41:02
cloud native integrations um but for food distributor we build up a solution
1:41:04
food distributor we build up a solution
1:41:04
food distributor we build up a solution where
1:41:05
where
1:41:05
where financial and operational data was being
1:41:08
financial and operational data was being
1:41:08
financial and operational data was being pushed into
1:41:09
pushed into
1:41:09
pushed into dynamics 365 islands and operations
1:41:11
dynamics 365 islands and operations
1:41:11
dynamics 365 islands and operations which is currently now
1:41:13
which is currently now
1:41:13
which is currently now finance and supply chain management
1:41:14
finance and supply chain management
1:41:14
finance and supply chain management still the same
1:41:16
still the same
1:41:16
still the same dynamic system but a different naming
1:41:18
dynamic system but a different naming
1:41:18
dynamic system but a different naming and we pushed some of that data that's
1:41:20
and we pushed some of that data that's
1:41:20
and we pushed some of that data that's being mapped a bit into or at least
1:41:22
being mapped a bit into or at least
1:41:22
being mapped a bit into or at least pushed into a logic app which hits as a
1:41:23
pushed into a logic app which hits as a
1:41:23
pushed into a logic app which hits as a barrier between those systems
1:41:25
barrier between those systems
1:41:25
barrier between those systems and then push the payload to what's
1:41:27
and then push the payload to what's
1:41:27
and then push the payload to what's called a recurring job
1:41:29
called a recurring job
1:41:29
called a recurring job there were no business event
1:41:30
there were no business event
1:41:30
there were no business event implementations back then so the
1:41:32
implementations back then so the
1:41:32
implementations back then so the id coming back from recurring job was
1:41:35
id coming back from recurring job was
1:41:35
id coming back from recurring job was pushed
1:41:35
pushed
1:41:36
pushed at least to another logic app that would
1:41:38
at least to another logic app that would
1:41:38
at least to another logic app that would just get the status of that recurring
1:41:39
just get the status of that recurring
1:41:39
just get the status of that recurring job and
1:41:40
job and
1:41:40
job and if it's finalized or not so this is kind
1:41:42
if it's finalized or not so this is kind
1:41:42
if it's finalized or not so this is kind of a hybrid solution
1:41:44
of a hybrid solution
1:41:44
of a hybrid solution where entrant monitoring being put in
1:41:46
where entrant monitoring being put in
1:41:46
where entrant monitoring being put in place where we leverage cosmos db
1:41:48
place where we leverage cosmos db
1:41:48
place where we leverage cosmos db as well so leverage all the metadata
1:41:50
as well so leverage all the metadata
1:41:50
as well so leverage all the metadata logging data centrally through a logic
1:41:52
logging data centrally through a logic
1:41:52
logging data centrally through a logic app
1:41:53
app
1:41:53
app into a cosmos db so kind of what worked
1:41:55
into a cosmos db so kind of what worked
1:41:56
into a cosmos db so kind of what worked is all the
1:41:56
is all the
1:41:56
is all the logging was being pushed into a cosmos
1:41:58
logging was being pushed into a cosmos
1:41:58
logging was being pushed into a cosmos db instance
1:42:00
db instance
1:42:00
db instance using a logic app it was indexed
1:42:02
using a logic app it was indexed
1:42:02
using a logic app it was indexed subsequently similar to
1:42:03
subsequently similar to
1:42:03
subsequently similar to the learning that from our previous
1:42:05
the learning that from our previous
1:42:05
the learning that from our previous project and then data could be
1:42:08
project and then data could be
1:42:08
project and then data could be shown through getting the data out of
1:42:10
shown through getting the data out of
1:42:10
shown through getting the data out of the cosmos db
1:42:12
the cosmos db
1:42:12
the cosmos db in combination with calling out dynamics
1:42:14
in combination with calling out dynamics
1:42:14
in combination with calling out dynamics with the job id
1:42:15
with the job id
1:42:15
with the job id or getting maybe some of the original
1:42:17
or getting maybe some of the original
1:42:17
or getting maybe some of the original mapped files from bistock server
1:42:20
mapped files from bistock server
1:42:20
mapped files from bistock server so kind of this is how one of those
1:42:22
so kind of this is how one of those
1:42:22
so kind of this is how one of those functional logs will look like so this
1:42:24
functional logs will look like so this
1:42:24
functional logs will look like so this is kind of
1:42:25
is kind of
1:42:25
is kind of one of the many loggings that were
1:42:27
one of the many loggings that were
1:42:28
one of the many loggings that were inside
1:42:28
inside
1:42:28
inside cosmos db as you can see of the way it's
1:42:31
cosmos db as you can see of the way it's
1:42:31
cosmos db as you can see of the way it's structured
1:42:33
structured
1:42:33
structured leading to the fact that you have this
1:42:34
leading to the fact that you have this
1:42:34
leading to the fact that you have this dashboard enabling the functional team
1:42:37
dashboard enabling the functional team
1:42:37
dashboard enabling the functional team that had to
1:42:37
that had to
1:42:38
that had to monitor the end-to-end process to look
1:42:39
monitor the end-to-end process to look
1:42:39
monitor the end-to-end process to look into
1:42:41
into
1:42:41
into the loggings of each of the process
1:42:43
the loggings of each of the process
1:42:43
the loggings of each of the process itself
1:42:44
itself
1:42:44
itself enabling them to use some query
1:42:46
enabling them to use some query
1:42:46
enabling them to use some query capability as well
1:42:49
capability as well
1:42:49
capability as well and drill down a little bit deeper into
1:42:52
and drill down a little bit deeper into
1:42:52
and drill down a little bit deeper into all of these steps and these steps are
1:42:54
all of these steps and these steps are
1:42:54
all of these steps and these steps are all data harnessed as documents inside
1:42:56
all data harnessed as documents inside
1:42:56
all data harnessed as documents inside document db and index
1:42:57
document db and index
1:42:58
document db and index and then also getting access to the met
1:43:00
and then also getting access to the met
1:43:00
and then also getting access to the met files through a function
1:43:03
files through a function
1:43:03
files through a function what we also created was kind of a
1:43:05
what we also created was kind of a
1:43:05
what we also created was kind of a functional alert
1:43:06
functional alert
1:43:06
functional alert meaning that we also leveraged the
1:43:08
meaning that we also leveraged the
1:43:08
meaning that we also leveraged the change feeds capability
1:43:09
change feeds capability
1:43:10
change feeds capability in cosmos they beat so some of those
1:43:12
in cosmos they beat so some of those
1:43:12
in cosmos they beat so some of those events
1:43:13
events
1:43:13
events or loggings were information coming back
1:43:16
or loggings were information coming back
1:43:16
or loggings were information coming back from dynamics
1:43:17
from dynamics
1:43:17
from dynamics indicating whether a job was filled or
1:43:19
indicating whether a job was filled or
1:43:19
indicating whether a job was filled or not if a job was filled so it's kind of
1:43:21
not if a job was filled so it's kind of
1:43:21
not if a job was filled so it's kind of an event sourcing looking at the state
1:43:23
an event sourcing looking at the state
1:43:23
an event sourcing looking at the state of each of these processes was
1:43:25
of each of these processes was
1:43:25
of each of these processes was filled you know process with an error or
1:43:27
filled you know process with an error or
1:43:27
filled you know process with an error or an error meaning that you know we want
1:43:29
an error meaning that you know we want
1:43:29
an error meaning that you know we want to
1:43:30
to
1:43:30
to isolate the information push it into a
1:43:33
isolate the information push it into a
1:43:33
isolate the information push it into a blob storage
1:43:34
blob storage
1:43:34
blob storage leading to a block created events or
1:43:35
leading to a block created events or
1:43:35
leading to a block created events or leveraging event grid
1:43:37
leveraging event grid
1:43:37
leveraging event grid getting all the metadata out and push
1:43:39
getting all the metadata out and push
1:43:39
getting all the metadata out and push that notification to the functional team
1:43:43
that notification to the functional team
1:43:43
that notification to the functional team and this led to the fact that the
1:43:44
and this led to the fact that the
1:43:44
and this led to the fact that the functional team will get a url as well
1:43:46
functional team will get a url as well
1:43:46
functional team will get a url as well straightaway directing them to what ran
1:43:49
straightaway directing them to what ran
1:43:49
straightaway directing them to what ran wrong from the
1:43:53
wrong from the
1:43:53
wrong from the processing of the recurring job and
1:43:55
processing of the recurring job and
1:43:55
processing of the recurring job and providing some details as well as you
1:43:56
providing some details as well as you
1:43:56
providing some details as well as you can see here
1:44:00
so to demonstrate a bit how this change
1:44:03
so to demonstrate a bit how this change
1:44:03
so to demonstrate a bit how this change feed capability works
1:44:06
feed capability works
1:44:06
feed capability works i kind of set up a demo so imagine you
1:44:08
i kind of set up a demo so imagine you
1:44:08
i kind of set up a demo so imagine you have your
1:44:10
have your
1:44:10
have your house instrumented with all kinds of
1:44:12
house instrumented with all kinds of
1:44:12
house instrumented with all kinds of sensor and devices and you've registered
1:44:14
sensor and devices and you've registered
1:44:14
sensor and devices and you've registered those with a service called iot hub
1:44:17
those with a service called iot hub
1:44:17
those with a service called iot hub the data subsequently is being streamed
1:44:19
the data subsequently is being streamed
1:44:19
the data subsequently is being streamed through event hubs to a function which
1:44:21
through event hubs to a function which
1:44:21
through event hubs to a function which pushes that data into a cosmos db which
1:44:24
pushes that data into a cosmos db which
1:44:24
pushes that data into a cosmos db which is kind of set as an event source
1:44:28
is kind of set as an event source
1:44:28
is kind of set as an event source also in this demo at least what i'm
1:44:30
also in this demo at least what i'm
1:44:30
also in this demo at least what i'm going to do is i'm going to mimic it the
1:44:32
going to do is i'm going to mimic it the
1:44:32
going to do is i'm going to mimic it the fact that the
1:44:34
fact that the
1:44:34
fact that the every change of one of the temperature
1:44:36
every change of one of the temperature
1:44:36
every change of one of the temperature of the room is also a
1:44:37
of the room is also a
1:44:37
of the room is also a you know a trigger for the function in
1:44:39
you know a trigger for the function in
1:44:39
you know a trigger for the function in with regards to a chains feed so there's
1:44:41
with regards to a chains feed so there's
1:44:41
with regards to a chains feed so there's an additional function
1:44:42
an additional function
1:44:42
an additional function if the temperature goes above a certain
1:44:46
if the temperature goes above a certain
1:44:46
if the temperature goes above a certain temperature then a message being pushed
1:44:50
temperature then a message being pushed
1:44:50
temperature then a message being pushed to
1:44:51
to
1:44:51
to a queue which has a logic app as a
1:44:53
a queue which has a logic app as a
1:44:53
a queue which has a logic app as a listener and then with logic apps
1:44:55
listener and then with logic apps
1:44:55
listener and then with logic apps there's many connectors you can leverage
1:44:56
there's many connectors you can leverage
1:44:56
there's many connectors you can leverage also for notifications like twilio
1:44:58
also for notifications like twilio
1:44:58
also for notifications like twilio exchange so i can send out a
1:45:00
exchange so i can send out a
1:45:00
exchange so i can send out a notification to my phone
1:45:01
notification to my phone
1:45:02
notification to my phone or somewhere else saying hey it's too
1:45:03
or somewhere else saying hey it's too
1:45:03
or somewhere else saying hey it's too hot
1:45:05
so i will move to my little setup so
1:45:10
so i will move to my little setup so
1:45:10
so i will move to my little setup so i just have a few of these entries
1:45:16
i just have a few of these entries
1:45:16
i just have a few of these entries of a few of the rooms let's say imagine
1:45:20
of a few of the rooms let's say imagine
1:45:20
of a few of the rooms let's say imagine in my house
1:45:22
in my house
1:45:22
in my house and i could for instance do
1:45:26
and i could for instance do
1:45:26
and i could for instance do an update
1:45:31
an update
1:45:31
an update and this would lead to the fact that a
1:45:33
and this would lead to the fact that a
1:45:33
and this would lead to the fact that a function will inspect
1:45:34
function will inspect
1:45:34
function will inspect at least get the trigger because of the
1:45:36
at least get the trigger because of the
1:45:36
at least get the trigger because of the chain speed and its binding
1:45:37
chain speed and its binding
1:45:38
chain speed and its binding will look at the actual
1:45:41
will look at the actual
1:45:41
will look at the actual temperature and if the temperature is
1:45:42
temperature and if the temperature is
1:45:42
temperature and if the temperature is above 25 celsius
1:45:44
above 25 celsius
1:45:44
above 25 celsius it will lead to the trigger of a logic
1:45:46
it will lead to the trigger of a logic
1:45:46
it will lead to the trigger of a logic app so if i move to my logic app
1:45:50
app so if i move to my logic app
1:45:50
app so if i move to my logic app and i've already ran this before the
1:45:52
and i've already ran this before the
1:45:52
and i've already ran this before the logic
1:45:53
logic
1:45:53
logic gets triggered because it listens to the
1:45:55
gets triggered because it listens to the
1:45:55
gets triggered because it listens to the queue
1:45:56
queue
1:45:56
queue it will get the document the change
1:45:58
it will get the document the change
1:45:58
it will get the document the change document
1:46:00
document
1:46:00
document in this case where the temperature has
1:46:02
in this case where the temperature has
1:46:02
in this case where the temperature has changed so it's changing then 25 not
1:46:05
changed so it's changing then 25 not
1:46:05
changed so it's changing then 25 not 29.5 celsius degrees meaning
1:46:09
29.5 celsius degrees meaning
1:46:09
29.5 celsius degrees meaning that it went over the threshold of 25
1:46:11
that it went over the threshold of 25
1:46:11
that it went over the threshold of 25 leading to the
1:46:12
leading to the
1:46:12
leading to the push of a message to a queue which led
1:46:15
push of a message to a queue which led
1:46:15
push of a message to a queue which led to the fact that
1:46:16
to the fact that
1:46:16
to the fact that you know this logic app is being
1:46:17
you know this logic app is being
1:46:17
you know this logic app is being triggered and it uses the sendgrid
1:46:19
triggered and it uses the sendgrid
1:46:19
triggered and it uses the sendgrid adapter so it's one of the many adapters
1:46:21
adapter so it's one of the many adapters
1:46:21
adapter so it's one of the many adapters allowing
1:46:22
allowing
1:46:22
allowing to send a message to my email account
1:46:26
to send a message to my email account
1:46:26
to send a message to my email account and this is the message i get back
1:46:28
and this is the message i get back
1:46:28
and this is the message i get back because and this shows me hey
1:46:30
because and this shows me hey
1:46:30
because and this shows me hey the temperature is hot so this kind of
1:46:31
the temperature is hot so this kind of
1:46:31
the temperature is hot so this kind of the signaling towards my
1:46:34
the signaling towards my
1:46:34
the signaling towards my um phone in this case or
1:46:38
email so
1:46:41
email so
1:46:41
email so in general this is just a simple
1:46:43
in general this is just a simple
1:46:43
in general this is just a simple showcase of
1:46:44
showcase of
1:46:44
showcase of of how the change is being leveraged and
1:46:46
of how the change is being leveraged and
1:46:46
of how the change is being leveraged and if i refer back to what i was talking
1:46:47
if i refer back to what i was talking
1:46:47
if i refer back to what i was talking about before for this customer
1:46:50
about before for this customer
1:46:50
about before for this customer all the all the inserts of
1:46:53
all the all the inserts of
1:46:53
all the all the inserts of the the the logging leads to all these
1:46:56
the the the logging leads to all these
1:46:56
the the the logging leads to all these chains feeds where there's a function
1:46:59
chains feeds where there's a function
1:46:59
chains feeds where there's a function behind it
1:46:59
behind it
1:47:00
behind it leveraging that cosmos db binding and
1:47:02
leveraging that cosmos db binding and
1:47:02
leveraging that cosmos db binding and then push out notifications in case
1:47:03
then push out notifications in case
1:47:03
then push out notifications in case there's an error
1:47:06
there's an error
1:47:06
there's an error and finally there's also a way of
1:47:09
and finally there's also a way of
1:47:09
and finally there's also a way of leveraging
1:47:10
leveraging
1:47:10
leveraging logic apps with cosmos db in a way of
1:47:13
logic apps with cosmos db in a way of
1:47:13
logic apps with cosmos db in a way of ingesting
1:47:14
ingesting
1:47:14
ingesting data so you can set up a simple pipeline
1:47:17
data so you can set up a simple pipeline
1:47:17
data so you can set up a simple pipeline when there's
1:47:17
when there's
1:47:17
when there's a simple data ingestion pipeline using
1:47:20
a simple data ingestion pipeline using
1:47:20
a simple data ingestion pipeline using logic apps when there's open exchange
1:47:22
logic apps when there's open exchange
1:47:22
logic apps when there's open exchange rate data for instance such as kind of
1:47:23
rate data for instance such as kind of
1:47:23
rate data for instance such as kind of currency data
1:47:25
currency data
1:47:25
currency data that you're gonna ingest and push into a
1:47:27
that you're gonna ingest and push into a
1:47:27
that you're gonna ingest and push into a cosmos db
1:47:28
cosmos db
1:47:28
cosmos db so this is kind of a setup this more pet
1:47:30
so this is kind of a setup this more pet
1:47:30
so this is kind of a setup this more pet project but it's derived from an actual
1:47:32
project but it's derived from an actual
1:47:32
project but it's derived from an actual use case at
1:47:33
use case at
1:47:33
use case at a customer where at an interval of a
1:47:36
a customer where at an interval of a
1:47:36
a customer where at an interval of a couple of hours data had to be ingested
1:47:38
couple of hours data had to be ingested
1:47:38
couple of hours data had to be ingested incorporated into
1:47:40
incorporated into
1:47:40
incorporated into a data store like a cosmos db to further
1:47:42
a data store like a cosmos db to further
1:47:42
a data store like a cosmos db to further processing and look into
1:47:44
processing and look into
1:47:44
processing and look into their way of providing services to their
1:47:47
their way of providing services to their
1:47:47
their way of providing services to their business
1:47:49
business
1:47:49
business and the way this looks and i will
1:47:50
and the way this looks and i will
1:47:50
and the way this looks and i will quickly go to my environment again
1:47:53
quickly go to my environment again
1:47:53
quickly go to my environment again so this kind of the setup completely you
1:47:54
so this kind of the setup completely you
1:47:54
so this kind of the setup completely you see all kinds of azure components
1:47:56
see all kinds of azure components
1:47:56
see all kinds of azure components including a cosmos db where all the
1:47:58
including a cosmos db where all the
1:47:58
including a cosmos db where all the currencies are being
1:47:59
currencies are being
1:47:59
currencies are being persisted
1:48:02
this will lead to um an execution of a
1:48:05
this will lead to um an execution of a
1:48:05
this will lead to um an execution of a logic app so this
1:48:06
logic app so this
1:48:06
logic app so this this demo has been running for quite a
1:48:07
this demo has been running for quite a
1:48:08
this demo has been running for quite a while and you can see kind of the output
1:48:11
while and you can see kind of the output
1:48:11
while and you can see kind of the output of the response from this open source
1:48:14
of the response from this open source
1:48:14
of the response from this open source open source exchange rate which is being
1:48:16
open source exchange rate which is being
1:48:16
open source exchange rate which is being pushed into
1:48:19
pushed into
1:48:19
pushed into document store i haven't got my cosmos
1:48:22
document store i haven't got my cosmos
1:48:22
document store i haven't got my cosmos db which is actually this one
1:48:25
db which is actually this one
1:48:25
db which is actually this one and you see there's a lot of entries um
1:48:28
and you see there's a lot of entries um
1:48:28
and you see there's a lot of entries um i think this is one the first i've been
1:48:30
i think this is one the first i've been
1:48:30
i think this is one the first i've been running this sample for quite a bit
1:48:31
running this sample for quite a bit
1:48:31
running this sample for quite a bit since
1:48:32
since
1:48:32
since 218. um i think currently
1:48:35
218. um i think currently
1:48:35
218. um i think currently we're at 22
1:48:39
we're at 22
1:48:39
we're at 22 367 entry so i'm going to use this data
1:48:41
367 entry so i'm going to use this data
1:48:41
367 entry so i'm going to use this data later on to do some analysis maybe and
1:48:43
later on to do some analysis maybe and
1:48:43
later on to do some analysis maybe and sign up some other so i'm going to
1:48:44
sign up some other so i'm going to
1:48:44
sign up some other so i'm going to extend it again with some other
1:48:46
extend it again with some other
1:48:46
extend it again with some other integration with cosmos db
1:48:48
integration with cosmos db
1:48:48
integration with cosmos db leading to more insights in how to
1:48:51
leading to more insights in how to
1:48:51
leading to more insights in how to leverage that data
1:48:55
so as you can see you can integrate with
1:48:59
so as you can see you can integrate with
1:48:59
so as you can see you can integrate with many azure kind of capabilities some are
1:49:01
many azure kind of capabilities some are
1:49:01
many azure kind of capabilities some are very interesting
1:49:03
very interesting
1:49:03
very interesting um i think search is one of the really
1:49:05
um i think search is one of the really
1:49:05
um i think search is one of the really valuable
1:49:07
valuable
1:49:07
valuable integrations with cosmos that be leading
1:49:09
integrations with cosmos that be leading
1:49:09
integrations with cosmos that be leading to quite some valuable
1:49:13
value to the business with regards to
1:49:14
value to the business with regards to
1:49:14
value to the business with regards to your solution you're leveraging with
1:49:15
your solution you're leveraging with
1:49:15
your solution you're leveraging with cosmos db
1:49:17
cosmos db
1:49:17
cosmos db um i've noticed that in these cases with
1:49:20
um i've noticed that in these cases with
1:49:20
um i've noticed that in these cases with every business but also with the whole
1:49:22
every business but also with the whole
1:49:22
every business but also with the whole food distributor that cosmos can be
1:49:23
food distributor that cosmos can be
1:49:23
food distributor that cosmos can be quite valuable
1:49:24
quite valuable
1:49:24
quite valuable and and also a vital component in your
1:49:27
and and also a vital component in your
1:49:27
and and also a vital component in your solution
1:49:27
solution
1:49:28
solution architecture but i also find that when i
1:49:30
architecture but i also find that when i
1:49:30
architecture but i also find that when i talked about
1:49:31
talked about
1:49:31
talked about some of these use cases some of the
1:49:33
some of these use cases some of the
1:49:33
some of these use cases some of the people or developers like yeah but
1:49:34
people or developers like yeah but
1:49:34
people or developers like yeah but cosmos db
1:49:35
cosmos db
1:49:35
cosmos db is very expensive and it's not the way
1:49:38
is very expensive and it's not the way
1:49:38
is very expensive and it's not the way you have to look like it you have to
1:49:39
you have to look like it you have to
1:49:39
you have to look like it you have to look at the added value of cosmos db
1:49:41
look at the added value of cosmos db
1:49:41
look at the added value of cosmos db itself
1:49:43
itself
1:49:43
itself with regards to the relics group they
1:49:45
with regards to the relics group they
1:49:45
with regards to the relics group they kind of had the subscription model which
1:49:46
kind of had the subscription model which
1:49:46
kind of had the subscription model which grossed in a lot of money with regards
1:49:48
grossed in a lot of money with regards
1:49:48
grossed in a lot of money with regards to the subscriptions and then based on
1:49:50
to the subscriptions and then based on
1:49:50
to the subscriptions and then based on the tco which was
1:49:51
the tco which was
1:49:51
the tco which was far less than the income it wasn't so
1:49:53
far less than the income it wasn't so
1:49:54
far less than the income it wasn't so expensive
1:49:54
expensive
1:49:54
expensive and with the whole view this literature
1:49:56
and with the whole view this literature
1:49:56
and with the whole view this literature we had a tco of our kind of monitoring
1:49:58
we had a tco of our kind of monitoring
1:49:58
we had a tco of our kind of monitoring solution in general including what we
1:50:00
solution in general including what we
1:50:00
solution in general including what we had the functional monitoring
1:50:01
had the functional monitoring
1:50:01
had the functional monitoring which was less than 200 euros a month
1:50:04
which was less than 200 euros a month
1:50:04
which was less than 200 euros a month but
1:50:05
but
1:50:05
but provided tremendous value for the
1:50:07
provided tremendous value for the
1:50:07
provided tremendous value for the operations
1:50:08
operations
1:50:08
operations and also the functional team behind
1:50:10
and also the functional team behind
1:50:10
and also the functional team behind dynamics 365.
1:50:14
dynamics 365.
1:50:14
dynamics 365. so um i'd like to thank you for
1:50:16
so um i'd like to thank you for
1:50:16
so um i'd like to thank you for attending this uh
1:50:17
attending this uh
1:50:17
attending this uh short but sweet session around
1:50:19
short but sweet session around
1:50:19
short but sweet session around integrating azure cosmos db
1:50:22
integrating azure cosmos db
1:50:22
integrating azure cosmos db with your other azure services so
1:50:25
with your other azure services so
1:50:25
with your other azure services so back to you guys for some q a
1:50:42
i think we're all stunned by your
1:50:43
i think we're all stunned by your
1:50:43
i think we're all stunned by your presentation
1:50:45
presentation
1:50:45
presentation oh really it's it was very good um
1:50:49
oh really it's it was very good um
1:50:49
oh really it's it was very good um yeah i i'm interested um
1:50:52
yeah i i'm interested um
1:50:52
yeah i i'm interested um in whether or not um you see
1:50:55
in whether or not um you see
1:50:55
in whether or not um you see uh any kind of ranking order between
1:50:58
uh any kind of ranking order between
1:50:58
uh any kind of ranking order between what what um
1:51:00
what what um
1:51:00
what what um um people prefer in the in these types
1:51:03
um people prefer in the in these types
1:51:03
um people prefer in the in these types of solutions
1:51:04
of solutions
1:51:04
of solutions uh or you know the particular approaches
1:51:06
uh or you know the particular approaches
1:51:06
uh or you know the particular approaches that people take
1:51:07
that people take
1:51:07
that people take uh when they're trying to make these
1:51:08
uh when they're trying to make these
1:51:08
uh when they're trying to make these composite architectures of you know
1:51:10
composite architectures of you know
1:51:10
composite architectures of you know cloud native
1:51:12
cloud native
1:51:12
cloud native uh uh services um is there are a patent
1:51:15
uh uh services um is there are a patent
1:51:15
uh uh services um is there are a patent there are
1:51:15
there are
1:51:15
there are popular patterns that you've seen
1:51:17
popular patterns that you've seen
1:51:17
popular patterns that you've seen yourself in the wild
1:51:19
yourself in the wild
1:51:19
yourself in the wild uh yeah it's definitely around what i
1:51:20
uh yeah it's definitely around what i
1:51:20
uh yeah it's definitely around what i talked about with regards to the uh the
1:51:23
talked about with regards to the uh the
1:51:23
talked about with regards to the uh the event sourcing
1:51:24
event sourcing
1:51:24
event sourcing enabling you to push documents into that
1:51:26
enabling you to push documents into that
1:51:26
enabling you to push documents into that store and then leveraging the change
1:51:27
store and then leveraging the change
1:51:27
store and then leveraging the change feeds which
1:51:29
feeds which
1:51:29
feeds which opens unlocks many types of solutions
1:51:31
opens unlocks many types of solutions
1:51:31
opens unlocks many types of solutions you can do in this case it was just
1:51:33
you can do in this case it was just
1:51:33
you can do in this case it was just about operations but it led to the fact
1:51:35
about operations but it led to the fact
1:51:35
about operations but it led to the fact that i we could do
1:51:36
that i we could do
1:51:36
that i we could do targeted emailing towards the team that
1:51:39
targeted emailing towards the team that
1:51:39
targeted emailing towards the team that is
1:51:40
is
1:51:40
is you know should get that information
1:51:42
you know should get that information
1:51:42
you know should get that information instead of generally
1:51:43
instead of generally
1:51:43
instead of generally having informational logging going to an
1:51:45
having informational logging going to an
1:51:45
having informational logging going to an i.t department where okay what do i do
1:51:47
i.t department where okay what do i do
1:51:47
i.t department where okay what do i do now dynamics is broken oh i don't know
1:51:49
now dynamics is broken oh i don't know
1:51:49
now dynamics is broken oh i don't know or dimension is missing or
1:51:51
or dimension is missing or
1:51:51
or dimension is missing or so yeah that's definitely one of the
1:51:53
so yeah that's definitely one of the
1:51:53
so yeah that's definitely one of the patterns i see a lot
1:51:55
patterns i see a lot
1:51:55
patterns i see a lot with regards to to cosmos db other being
1:51:59
with regards to to cosmos db other being
1:51:59
with regards to to cosmos db other being storing data and leveraging search which
1:52:01
storing data and leveraging search which
1:52:01
storing data and leveraging search which is a great combination i must say
1:52:04
is a great combination i must say
1:52:04
is a great combination i must say yeah i guess one of the
1:52:07
yeah i guess one of the
1:52:07
yeah i guess one of the uh question is well we just we just had
1:52:09
uh question is well we just we just had
1:52:09
uh question is well we just we just had the uh the cost optimization video flash
1:52:13
the uh the cost optimization video flash
1:52:13
the uh the cost optimization video flash before but i wondered if you had any any
1:52:15
before but i wondered if you had any any
1:52:15
before but i wondered if you had any any uh
1:52:16
uh
1:52:16
uh tips or extra thoughts around uh how
1:52:18
tips or extra thoughts around uh how
1:52:18
tips or extra thoughts around uh how people can
1:52:19
people can
1:52:19
people can uh can approach this because as you said
1:52:21
uh can approach this because as you said
1:52:21
uh can approach this because as you said we have had this uh
1:52:23
we have had this uh
1:52:23
we have had this uh this outrageous accusation at times
1:52:26
this outrageous accusation at times
1:52:26
this outrageous accusation at times yeah i know yeah well
1:52:30
yeah i know yeah well
1:52:30
yeah i know yeah well yeah well we did some performance
1:52:31
yeah well we did some performance
1:52:31
yeah well we did some performance testing i think mark brown also created
1:52:33
testing i think mark brown also created
1:52:33
testing i think mark brown also created some really great
1:52:34
some really great
1:52:34
some really great code to do some optimizations and
1:52:36
code to do some optimizations and
1:52:36
code to do some optimizations and performance with regards to your
1:52:38
performance with regards to your
1:52:38
performance with regards to your customers to be determining what kind of
1:52:39
customers to be determining what kind of
1:52:39
customers to be determining what kind of rus you need
1:52:41
rus you need
1:52:41
rus you need especially with these customers that i
1:52:43
especially with these customers that i
1:52:43
especially with these customers that i worked with we also very looked at
1:52:45
worked with we also very looked at
1:52:45
worked with we also very looked at cost and how to dimension or at least
1:52:48
cost and how to dimension or at least
1:52:48
cost and how to dimension or at least how to set the ru's
1:52:49
how to set the ru's
1:52:49
how to set the ru's we saw that workloads with regards to
1:52:51
we saw that workloads with regards to
1:52:51
we saw that workloads with regards to that hybrid scenario wasn't that high
1:52:53
that hybrid scenario wasn't that high
1:52:53
that hybrid scenario wasn't that high leading to i would say minimal costs
1:52:57
leading to i would say minimal costs
1:52:57
leading to i would say minimal costs with regards to the setup for
1:53:00
with regards to the setup for
1:53:00
with regards to the setup for a knowledge base we noticed that
1:53:01
a knowledge base we noticed that
1:53:02
a knowledge base we noticed that initially when you do your loads
1:53:03
initially when you do your loads
1:53:03
initially when you do your loads you need to bump up the ru's but then
1:53:05
you need to bump up the ru's but then
1:53:05
you need to bump up the ru's but then you slowly can decrease them because
1:53:07
you slowly can decrease them because
1:53:07
you slowly can decrease them because then all your content is in your data
1:53:09
then all your content is in your data
1:53:09
then all your content is in your data models right
1:53:10
models right
1:53:10
models right and then it depends more on the traffic
1:53:12
and then it depends more on the traffic
1:53:12
and then it depends more on the traffic um so you have to dynamically sometimes
1:53:14
um so you have to dynamically sometimes
1:53:14
um so you have to dynamically sometimes set your arrows depending if
1:53:17
set your arrows depending if
1:53:17
set your arrows depending if uh during business hour for instance
1:53:18
uh during business hour for instance
1:53:18
uh during business hour for instance there's more people going to a
1:53:20
there's more people going to a
1:53:20
there's more people going to a let's say a knowledge base than in the
1:53:21
let's say a knowledge base than in the
1:53:21
let's say a knowledge base than in the night right so you have to look at those
1:53:23
night right so you have to look at those
1:53:23
night right so you have to look at those dimension as well
1:53:25
dimension as well
1:53:25
dimension as well so i think monitoring traffic so we
1:53:27
so i think monitoring traffic so we
1:53:27
so i think monitoring traffic so we definitely looked at traffic especially
1:53:29
definitely looked at traffic especially
1:53:29
definitely looked at traffic especially at the relics
1:53:30
at the relics
1:53:30
at the relics we looked at traffic so when can we
1:53:31
we looked at traffic so when can we
1:53:31
we looked at traffic so when can we expect more traffic and we can spec less
1:53:33
expect more traffic and we can spec less
1:53:33
expect more traffic and we can spec less and with the hybrid solution it's just
1:53:35
and with the hybrid solution it's just
1:53:35
and with the hybrid solution it's just that we knew the numbers we knew how
1:53:36
that we knew the numbers we knew how
1:53:36
that we knew the numbers we knew how many traffic that was going to be
1:53:38
many traffic that was going to be
1:53:38
many traffic that was going to be generated during the
1:53:39
generated during the
1:53:39
generated during the the month so there were a few
1:53:41
the month so there were a few
1:53:41
the month so there were a few operational data
1:53:43
operational data
1:53:43
operational data entries during the day and with
1:53:46
entries during the day and with
1:53:46
entries during the day and with financial data it was just once a month
1:53:48
financial data it was just once a month
1:53:48
financial data it was just once a month so you know exactly when to expect so
1:53:50
so you know exactly when to expect so
1:53:50
so you know exactly when to expect so you definitely have to look at traffic
1:53:51
you definitely have to look at traffic
1:53:51
you definitely have to look at traffic as well
1:53:52
as well
1:53:52
as well with regards to our use and the usage
1:53:55
with regards to our use and the usage
1:53:55
with regards to our use and the usage and the workloads of your
1:53:56
and the workloads of your
1:53:56
and the workloads of your cosmos db yeah i think everybody should
1:53:59
cosmos db yeah i think everybody should
1:53:59
cosmos db yeah i think everybody should take note of your
1:54:00
take note of your
1:54:00
take note of your twitter handle because they'll be uh
1:54:02
twitter handle because they'll be uh
1:54:02
twitter handle because they'll be uh asking you for tips
1:54:05
asking you for tips
1:54:05
asking you for tips that's really great yeah engaging with
1:54:06
that's really great yeah engaging with
1:54:06
that's really great yeah engaging with the cloud economy is
1:54:08
the cloud economy is
1:54:08
the cloud economy is as well as um uh you know understanding
1:54:10
as well as um uh you know understanding
1:54:10
as well as um uh you know understanding the different offers uh that we saw
1:54:12
the different offers uh that we saw
1:54:12
the different offers uh that we saw before in that video
1:54:13
before in that video
1:54:13
before in that video um do we have any other questions
1:54:17
um do we have any other questions
1:54:17
um do we have any other questions out there yeah one question that came up
1:54:20
out there yeah one question that came up
1:54:20
out there yeah one question that came up was do you ever find yourself using the
1:54:22
was do you ever find yourself using the
1:54:22
was do you ever find yourself using the different
1:54:22
different
1:54:22
different uh forms of um provisioning models for
1:54:26
uh forms of um provisioning models for
1:54:26
uh forms of um provisioning models for for cost optimization such as
1:54:29
for cost optimization such as
1:54:29
for cost optimization such as provision throughput or auto scale or
1:54:32
provision throughput or auto scale or
1:54:32
provision throughput or auto scale or serverless
1:54:34
um going forward
1:54:37
um going forward
1:54:37
um going forward yes certainly uh but initially when i
1:54:39
yes certainly uh but initially when i
1:54:39
yes certainly uh but initially when i was doing this in 217 to 18 there were
1:54:41
was doing this in 217 to 18 there were
1:54:41
was doing this in 217 to 18 there were no serverless
1:54:42
no serverless
1:54:42
no serverless there were less capabilities back then
1:54:44
there were less capabilities back then
1:54:44
there were less capabilities back then than there are today so
1:54:46
than there are today so
1:54:46
than there are today so cosmos db has definitely matured in that
1:54:48
cosmos db has definitely matured in that
1:54:48
cosmos db has definitely matured in that way and these capabilities of force are
1:54:50
way and these capabilities of force are
1:54:50
way and these capabilities of force are currently available at least for a while
1:54:53
currently available at least for a while
1:54:53
currently available at least for a while but back in 2018 when i started out in
1:54:56
but back in 2018 when i started out in
1:54:56
but back in 2018 when i started out in the
1:54:56
the
1:54:56
the beginning of 219 they were less
1:54:58
beginning of 219 they were less
1:54:58
beginning of 219 they were less available so
1:55:00
available so
1:55:00
available so i think the cosmodb did a great team in
1:55:02
i think the cosmodb did a great team in
1:55:02
i think the cosmodb did a great team in expanding and evolving the service also
1:55:05
expanding and evolving the service also
1:55:05
expanding and evolving the service also in different models
1:55:06
in different models
1:55:06
in different models enabling also people with serverless to
1:55:08
enabling also people with serverless to
1:55:08
enabling also people with serverless to see how the service work without you
1:55:09
see how the service work without you
1:55:09
see how the service work without you know getting this
1:55:10
know getting this
1:55:10
know getting this large amount of costs so
1:55:13
large amount of costs so
1:55:13
large amount of costs so to say cool
1:55:17
to say cool
1:55:17
to say cool well thank you so much for the
1:55:18
well thank you so much for the
1:55:18
well thank you so much for the presentation i really enjoyed you're
1:55:20
presentation i really enjoyed you're
1:55:20
presentation i really enjoyed you're welcome
1:55:20
welcome
1:55:20
welcome okay thank you i'd like to introduce
1:55:24
okay thank you i'd like to introduce
1:55:24
okay thank you i'd like to introduce our next speaker um his name
1:55:27
our next speaker um his name
1:55:27
our next speaker um his name is javi and he'll be speaking about uh
1:55:30
is javi and he'll be speaking about uh
1:55:30
is javi and he'll be speaking about uh his company and his journey
1:55:32
his company and his journey
1:55:32
his company and his journey uh going through uh using azure cosmos
1:55:35
uh going through uh using azure cosmos
1:55:35
uh going through uh using azure cosmos tv
1:55:37
tv
1:55:38
tv hi there thank you
1:55:41
hi there thank you
1:55:41
hi there thank you okay so let's get started hi everyone
1:55:43
okay so let's get started hi everyone
1:55:43
okay so let's get started hi everyone good morning good afternoon good evening
1:55:45
good morning good afternoon good evening
1:55:45
good morning good afternoon good evening and welcome to my session i hope you're
1:55:47
and welcome to my session i hope you're
1:55:47
and welcome to my session i hope you're enjoying
1:55:48
enjoying
1:55:48
enjoying the conference so far my name is javier
1:55:51
the conference so far my name is javier
1:55:51
the conference so far my name is javier velasco and i work as an api engineering
1:55:53
velasco and i work as an api engineering
1:55:53
velasco and i work as an api engineering lead
1:55:54
lead
1:55:54
lead at burcell in case you're not familiar
1:55:56
at burcell in case you're not familiar
1:55:56
at burcell in case you're not familiar with what purcell is
1:55:58
with what purcell is
1:55:58
with what purcell is versailles is simply the fastest way to
1:56:00
versailles is simply the fastest way to
1:56:00
versailles is simply the fastest way to deploy front-end applications
1:56:02
deploy front-end applications
1:56:02
deploy front-end applications we enabled collaboration by giving
1:56:03
we enabled collaboration by giving
1:56:03
we enabled collaboration by giving front-end developers comprehensive tools
1:56:06
front-end developers comprehensive tools
1:56:06
front-end developers comprehensive tools to build high-quality
1:56:08
to build high-quality
1:56:08
to build high-quality websites our mission is to help build a
1:56:11
websites our mission is to help build a
1:56:11
websites our mission is to help build a better web
1:56:12
better web
1:56:12
better web with burcell you can connect a
1:56:13
with burcell you can connect a
1:56:13
with burcell you can connect a repository to a git provider
1:56:16
repository to a git provider
1:56:16
repository to a git provider and instantly generate multi-cloud
1:56:17
and instantly generate multi-cloud
1:56:18
and instantly generate multi-cloud deployments that are globally available
1:56:20
deployments that are globally available
1:56:20
deployments that are globally available through our edge network with every git
1:56:22
through our edge network with every git
1:56:22
through our edge network with every git push
1:56:23
push
1:56:23
push we have been using cosmos since 2017 as
1:56:27
we have been using cosmos since 2017 as
1:56:28
we have been using cosmos since 2017 as our main persistency system and it has
1:56:30
our main persistency system and it has
1:56:30
our main persistency system and it has been it has been a really big part of
1:56:32
been it has been a really big part of
1:56:32
been it has been a really big part of our history since then
1:56:34
our history since then
1:56:34
our history since then uh being our main persistency we have uh
1:56:37
uh being our main persistency we have uh
1:56:37
uh being our main persistency we have uh we have had a very intensive usage of it
1:56:39
we have had a very intensive usage of it
1:56:39
we have had a very intensive usage of it for both our apis and the edge network
1:56:41
for both our apis and the edge network
1:56:41
for both our apis and the edge network and in the case of the edge network
1:56:43
and in the case of the edge network
1:56:44
and in the case of the edge network latency is especially critical and
1:56:46
latency is especially critical and
1:56:46
latency is especially critical and cosmos distributed nature has been a
1:56:48
cosmos distributed nature has been a
1:56:48
cosmos distributed nature has been a great fit for
1:56:49
great fit for
1:56:49
great fit for it during all of this time cosmos db
1:56:51
it during all of this time cosmos db
1:56:52
it during all of this time cosmos db helps us scale tremendously well from
1:56:54
helps us scale tremendously well from
1:56:54
helps us scale tremendously well from just a few
1:56:54
just a few
1:56:54
just a few millions requests per week to more than
1:56:58
millions requests per week to more than
1:56:58
millions requests per week to more than 10 billion requests a week that we're
1:57:00
10 billion requests a week that we're
1:57:00
10 billion requests a week that we're handling today
1:57:01
handling today
1:57:01
handling today overall we are currently making more
1:57:03
overall we are currently making more
1:57:03
overall we are currently making more than 1.7 billion requests a day to the
1:57:06
than 1.7 billion requests a day to the
1:57:06
than 1.7 billion requests a day to the database and that's all
1:57:07
database and that's all
1:57:07
database and that's all scaled smoothly keeping always a very
1:57:10
scaled smoothly keeping always a very
1:57:10
scaled smoothly keeping always a very good latency
1:57:12
good latency
1:57:12
good latency so in this talk we will see how we used
1:57:15
so in this talk we will see how we used
1:57:15
so in this talk we will see how we used cosmos at scaled in burcell to work in a
1:57:17
cosmos at scaled in burcell to work in a
1:57:17
cosmos at scaled in burcell to work in a very performant way
1:57:19
very performant way
1:57:19
very performant way and there is no better way to illustrate
1:57:20
and there is no better way to illustrate
1:57:20
and there is no better way to illustrate the journey that we have had with cosmos
1:57:22
the journey that we have had with cosmos
1:57:22
the journey that we have had with cosmos than
1:57:23
than
1:57:23
than walking you through the wrapper that we
1:57:24
walking you through the wrapper that we
1:57:24
walking you through the wrapper that we have built on top of the node sdk
1:57:27
have built on top of the node sdk
1:57:27
have built on top of the node sdk to optimize our access patterns it
1:57:29
to optimize our access patterns it
1:57:29
to optimize our access patterns it started as a way to put together some
1:57:31
started as a way to put together some
1:57:31
started as a way to put together some code patterns that we required in a
1:57:33
code patterns that we required in a
1:57:33
code patterns that we required in a day-to-day basis but later
1:57:35
day-to-day basis but later
1:57:35
day-to-day basis but later it evolved into something more complex
1:57:36
it evolved into something more complex
1:57:36
it evolved into something more complex that helped us to use cosmos at scale
1:57:39
that helped us to use cosmos at scale
1:57:39
that helped us to use cosmos at scale and solve problems that
1:57:40
and solve problems that
1:57:40
and solve problems that you might be facing today as well this
1:57:43
you might be facing today as well this
1:57:43
you might be facing today as well this is a talk that i expect to be very
1:57:45
is a talk that i expect to be very
1:57:45
is a talk that i expect to be very useful for developers
1:57:47
useful for developers
1:57:47
useful for developers we are running a lot of api services on
1:57:49
we are running a lot of api services on
1:57:49
we are running a lot of api services on kubernetes making an extensive use of
1:57:51
kubernetes making an extensive use of
1:57:51
kubernetes making an extensive use of typescript
1:57:51
typescript
1:57:52
typescript and node.js and of course the cis if
1:57:54
and node.js and of course the cis if
1:57:54
and node.js and of course the cis if you're a developer working with cosmos
1:57:56
you're a developer working with cosmos
1:57:56
you're a developer working with cosmos and these technologies i'm pretty sure
1:57:57
and these technologies i'm pretty sure
1:57:58
and these technologies i'm pretty sure you're going to find this useful
1:57:59
you're going to find this useful
1:57:59
you're going to find this useful nonetheless i expect you to find it
1:58:01
nonetheless i expect you to find it
1:58:01
nonetheless i expect you to find it interesting as well if you are a devops
1:58:03
interesting as well if you are a devops
1:58:03
interesting as well if you are a devops sre or if you're interested in these
1:58:05
sre or if you're interested in these
1:58:05
sre or if you're interested in these technologies
1:58:06
technologies
1:58:06
technologies so we are going to divide this talk into
1:58:09
so we are going to divide this talk into
1:58:09
so we are going to divide this talk into a few sections that can put together
1:58:11
a few sections that can put together
1:58:11
a few sections that can put together some of the features that we
1:58:13
some of the features that we
1:58:13
some of the features that we have been building over time we will
1:58:15
have been building over time we will
1:58:15
have been building over time we will start by defining defining the reason
1:58:17
start by defining defining the reason
1:58:17
start by defining defining the reason why we have an sdk rubber
1:58:19
why we have an sdk rubber
1:58:19
why we have an sdk rubber and then we're going to quickly take a
1:58:20
and then we're going to quickly take a
1:58:20
and then we're going to quickly take a look into our observability tooling
1:58:22
look into our observability tooling
1:58:22
look into our observability tooling which includes
1:58:23
which includes
1:58:23
which includes some primitive features that we've
1:58:25
some primitive features that we've
1:58:25
some primitive features that we've introduced back in 2017
1:58:28
introduced back in 2017
1:58:28
introduced back in 2017 to more recent and sophisticated tracing
1:58:30
to more recent and sophisticated tracing
1:58:30
to more recent and sophisticated tracing metrics
1:58:31
metrics
1:58:31
metrics next we're going to see how we improve
1:58:33
next we're going to see how we improve
1:58:33
next we're going to see how we improve performance and reliability with some
1:58:34
performance and reliability with some
1:58:34
performance and reliability with some middleware like features for our cosmos
1:58:36
middleware like features for our cosmos
1:58:36
middleware like features for our cosmos calls
1:58:37
calls
1:58:37
calls and then we will see how we enrich our
1:58:39
and then we will see how we enrich our
1:58:39
and then we will see how we enrich our data models introducing
1:58:40
data models introducing
1:58:40
data models introducing a configuration layer that has been
1:58:43
a configuration layer that has been
1:58:43
a configuration layer that has been pretty much the cornerstone to more
1:58:45
pretty much the cornerstone to more
1:58:45
pretty much the cornerstone to more advanced tooling
1:58:46
advanced tooling
1:58:46
advanced tooling and finally we will close with some
1:58:48
and finally we will close with some
1:58:48
and finally we will close with some slightly more complex tools that we
1:58:50
slightly more complex tools that we
1:58:50
slightly more complex tools that we created
1:58:50
created
1:58:50
created to optimize access and guarantee
1:58:52
to optimize access and guarantee
1:58:52
to optimize access and guarantee cross-partition uniqueness of certain
1:58:55
cross-partition uniqueness of certain
1:58:55
cross-partition uniqueness of certain values that we called
1:58:56
values that we called
1:58:56
values that we called secondary keys finally we will introduce
1:58:59
secondary keys finally we will introduce
1:58:59
secondary keys finally we will introduce a feature we called
1:59:00
a feature we called
1:59:00
a feature we called mirroring that helps us a lot when it
1:59:02
mirroring that helps us a lot when it
1:59:02
mirroring that helps us a lot when it comes to
1:59:03
comes to
1:59:03
comes to repetition collections so let's get
1:59:06
repetition collections so let's get
1:59:06
repetition collections so let's get started wire wrapper
1:59:07
started wire wrapper
1:59:07
started wire wrapper right in this is a typical scenario of
1:59:11
right in this is a typical scenario of
1:59:11
right in this is a typical scenario of any application that is using vendor
1:59:13
any application that is using vendor
1:59:13
any application that is using vendor libraries or sdks
1:59:14
libraries or sdks
1:59:14
libraries or sdks you're in control of your software but
1:59:16
you're in control of your software but
1:59:16
you're in control of your software but not really in control of the libraries
1:59:18
not really in control of the libraries
1:59:18
not really in control of the libraries it's true that you can browse them you
1:59:19
it's true that you can browse them you
1:59:19
it's true that you can browse them you can fork them you can pr
1:59:21
can fork them you can pr
1:59:21
can fork them you can pr but changes that uh but you can't make
1:59:24
but changes that uh but you can't make
1:59:24
but changes that uh but you can't make changes that
1:59:24
changes that
1:59:24
changes that are specific to your domain forking is
1:59:27
are specific to your domain forking is
1:59:27
are specific to your domain forking is usually not the best idea if you want to
1:59:29
usually not the best idea if you want to
1:59:29
usually not the best idea if you want to keep
1:59:29
keep
1:59:30
keep leveraging maintainers patches and
1:59:32
leveraging maintainers patches and
1:59:32
leveraging maintainers patches and updates anyway
1:59:33
updates anyway
1:59:33
updates anyway so this means that you are in the end
1:59:35
so this means that you are in the end
1:59:35
so this means that you are in the end not really in control to do whatever you
1:59:37
not really in control to do whatever you
1:59:37
not really in control to do whatever you want to do with that code
1:59:38
want to do with that code
1:59:38
want to do with that code and in our case we have plenty of
1:59:40
and in our case we have plenty of
1:59:40
and in our case we have plenty of services that are using the sdk
1:59:43
services that are using the sdk
1:59:43
services that are using the sdk directly so we would be sold to these
1:59:45
directly so we would be sold to these
1:59:45
directly so we would be sold to these places which would be
1:59:47
places which would be
1:59:47
places which would be future changes with this model and
1:59:50
future changes with this model and
1:59:50
future changes with this model and we have to also consider that there
1:59:52
we have to also consider that there
1:59:52
we have to also consider that there might be cases where you want to add
1:59:53
might be cases where you want to add
1:59:53
might be cases where you want to add some
1:59:54
some
1:59:54
some extra processing layer to each sdk
1:59:56
extra processing layer to each sdk
1:59:56
extra processing layer to each sdk operation
1:59:57
operation
1:59:57
operation so this is for this that we wanted to
1:59:59
so this is for this that we wanted to
1:59:59
so this is for this that we wanted to transition from this model
2:00:01
transition from this model
2:00:01
transition from this model to this other model we would have an
2:00:03
to this other model we would have an
2:00:03
to this other model we would have an adapter between our services and the sdk
2:00:06
adapter between our services and the sdk
2:00:06
adapter between our services and the sdk and that's going to dramatically reduce
2:00:08
and that's going to dramatically reduce
2:00:08
and that's going to dramatically reduce the amount of places where we have to
2:00:09
the amount of places where we have to
2:00:09
the amount of places where we have to make changes in the events of an
2:00:11
make changes in the events of an
2:00:11
make changes in the events of an sdk breaking change we already leave
2:00:13
sdk breaking change we already leave
2:00:14
sdk breaking change we already leave this when we went through some major
2:00:15
this when we went through some major
2:00:15
this when we went through some major updates of the sdk
2:00:17
updates of the sdk
2:00:17
updates of the sdk for example when cosmos 3 was released
2:00:19
for example when cosmos 3 was released
2:00:19
for example when cosmos 3 was released it was shipped with some breaking
2:00:20
it was shipped with some breaking
2:00:20
it was shipped with some breaking changes of
2:00:21
changes of
2:00:21
changes of optimization that we could very easily
2:00:23
optimization that we could very easily
2:00:24
optimization that we could very easily leverage with a very small effort since
2:00:26
leverage with a very small effort since
2:00:26
leverage with a very small effort since we just needed to change our adapter
2:00:28
we just needed to change our adapter
2:00:28
we just needed to change our adapter another huge benefit of this model is
2:00:30
another huge benefit of this model is
2:00:30
another huge benefit of this model is that it makes very easy to inject
2:00:32
that it makes very easy to inject
2:00:32
that it makes very easy to inject layers on behavior on top of every
2:00:35
layers on behavior on top of every
2:00:35
layers on behavior on top of every database call
2:00:36
database call
2:00:36
database call that is why because uh that i mean
2:00:40
that is why because uh that i mean
2:00:40
that is why because uh that i mean that is why we adapted every single
2:00:42
that is why we adapted every single
2:00:42
that is why we adapted every single cosmos
2:00:43
cosmos
2:00:43
cosmos code to include layers that bring
2:00:45
code to include layers that bring
2:00:45
code to include layers that bring amongst other things
2:00:46
amongst other things
2:00:46
amongst other things observability our observability pipeline
2:00:50
observability our observability pipeline
2:00:50
observability our observability pipeline includes very simple tooling like
2:00:52
includes very simple tooling like
2:00:52
includes very simple tooling like logging statsd metrics
2:00:54
logging statsd metrics
2:00:54
logging statsd metrics both introducing in 2017
2:00:57
both introducing in 2017
2:00:57
both introducing in 2017 and also includes some more complex
2:00:58
and also includes some more complex
2:00:58
and also includes some more complex tooling like tracing which we
2:01:00
tooling like tracing which we
2:01:00
tooling like tracing which we implemented
2:01:01
implemented
2:01:01
implemented using apm now that we know that
2:01:04
using apm now that we know that
2:01:04
using apm now that we know that every single sdk wrapper is wrapped we
2:01:07
every single sdk wrapper is wrapped we
2:01:07
every single sdk wrapper is wrapped we can
2:01:08
can
2:01:08
can introduce how we pluck in these features
2:01:11
introduce how we pluck in these features
2:01:11
introduce how we pluck in these features so assuming that you can generalize the
2:01:14
so assuming that you can generalize the
2:01:14
so assuming that you can generalize the contract of
2:01:14
contract of
2:01:14
contract of every cosmos call to a lazy evaluated
2:01:17
every cosmos call to a lazy evaluated
2:01:17
every cosmos call to a lazy evaluated function
2:01:18
function
2:01:18
function that would end up resolving a response
2:01:20
that would end up resolving a response
2:01:20
that would end up resolving a response or a feed response we can create a
2:01:22
or a feed response we can create a
2:01:22
or a feed response we can create a composition of higher order functions
2:01:24
composition of higher order functions
2:01:24
composition of higher order functions that would be adding layers with very
2:01:26
that would be adding layers with very
2:01:26
that would be adding layers with very specific behavior
2:01:28
specific behavior
2:01:28
specific behavior to every sdk operation for example in
2:01:31
to every sdk operation for example in
2:01:31
to every sdk operation for example in this case in the snippet what we are
2:01:32
this case in the snippet what we are
2:01:32
this case in the snippet what we are doing is bringing observability
2:01:34
doing is bringing observability
2:01:34
doing is bringing observability through two decorators one is with stats
2:01:37
through two decorators one is with stats
2:01:37
through two decorators one is with stats emitter
2:01:39
emitter
2:01:39
emitter that would result would receive an event
2:01:41
that would result would receive an event
2:01:41
that would result would receive an event emitter through
2:01:42
emitter through
2:01:42
emitter through some provided configuration and the
2:01:44
some provided configuration and the
2:01:44
some provided configuration and the decorator would use it to
2:01:46
decorator would use it to
2:01:46
decorator would use it to publish events with metadata for example
2:01:48
publish events with metadata for example
2:01:48
publish events with metadata for example you would tell
2:01:50
you would tell
2:01:50
you would tell how much time and a period an operation
2:01:52
how much time and a period an operation
2:01:52
how much time and a period an operation took
2:01:53
took
2:01:53
took what is the query that it run time
2:01:55
what is the query that it run time
2:01:55
what is the query that it run time taking cost etc
2:01:58
taking cost etc
2:01:58
taking cost etc with logging is a much more easier one
2:02:00
with logging is a much more easier one
2:02:00
with logging is a much more easier one and the responsibility would be to
2:02:02
and the responsibility would be to
2:02:02
and the responsibility would be to simply log a line
2:02:03
simply log a line
2:02:03
simply log a line to std out before and after the
2:02:05
to std out before and after the
2:02:05
to std out before and after the operation happens
2:02:06
operation happens
2:02:06
operation happens this dog is going to be alerting on the
2:02:09
this dog is going to be alerting on the
2:02:09
this dog is going to be alerting on the cases
2:02:10
cases
2:02:10
cases where it becomes very slow or there are
2:02:12
where it becomes very slow or there are
2:02:12
where it becomes very slow or there are expensive
2:02:13
expensive
2:02:13
expensive queries or failures so we can relate
2:02:15
queries or failures so we can relate
2:02:15
queries or failures so we can relate stack traces
2:02:16
stack traces
2:02:16
stack traces in our logs to potentially harming
2:02:19
in our logs to potentially harming
2:02:19
in our logs to potentially harming cosmos operations
2:02:21
cosmos operations
2:02:21
cosmos operations the status emitter in particular was
2:02:23
the status emitter in particular was
2:02:23
the status emitter in particular was very useful
2:02:24
very useful
2:02:24
very useful because it allowed us to have a
2:02:26
because it allowed us to have a
2:02:26
because it allowed us to have a complementary paneling
2:02:27
complementary paneling
2:02:27
complementary paneling or in our observability pipeline that is
2:02:30
or in our observability pipeline that is
2:02:30
or in our observability pipeline that is showing pretty much the same as the
2:02:32
showing pretty much the same as the
2:02:32
showing pretty much the same as the cosmos usage panel is giving in the
2:02:34
cosmos usage panel is giving in the
2:02:34
cosmos usage panel is giving in the portal but with a very important
2:02:36
portal but with a very important
2:02:36
portal but with a very important difference which is that
2:02:37
difference which is that
2:02:38
difference which is that in this case we can define very grain
2:02:40
in this case we can define very grain
2:02:40
in this case we can define very grain fine-grained alerts
2:02:42
fine-grained alerts
2:02:42
fine-grained alerts when it comes to root consumption for
2:02:44
when it comes to root consumption for
2:02:44
when it comes to root consumption for example if it's going above
2:02:45
example if it's going above
2:02:45
example if it's going above a certain limit we can set an alert or
2:02:47
a certain limit we can set an alert or
2:02:47
a certain limit we can set an alert or if we have a very expensive
2:02:49
if we have a very expensive
2:02:49
if we have a very expensive or slow operation we can also trigger
2:02:52
or slow operation we can also trigger
2:02:52
or slow operation we can also trigger alerts nonetheless this is still working
2:02:55
alerts nonetheless this is still working
2:02:55
alerts nonetheless this is still working on top of the sdk
2:02:57
on top of the sdk
2:02:57
on top of the sdk request but not from the requests
2:02:59
request but not from the requests
2:02:59
request but not from the requests themselves so we are still blind and we
2:03:01
themselves so we are still blind and we
2:03:01
themselves so we are still blind and we don't know
2:03:02
don't know
2:03:02
don't know how much time is being spent on the sdk
2:03:04
how much time is being spent on the sdk
2:03:04
how much time is being spent on the sdk how much time is spending or wrapper how
2:03:06
how much time is spending or wrapper how
2:03:06
how much time is spending or wrapper how much time that request
2:03:07
much time that request
2:03:07
much time that request actually took so it was at this stage
2:03:11
actually took so it was at this stage
2:03:11
actually took so it was at this stage when it became very useful a new feature
2:03:13
when it became very useful a new feature
2:03:13
when it became very useful a new feature that i think is still undocumented in
2:03:15
that i think is still undocumented in
2:03:15
that i think is still undocumented in the node sdk
2:03:17
the node sdk
2:03:17
the node sdk which is the plugins that has that is
2:03:19
which is the plugins that has that is
2:03:19
which is the plugins that has that is going to allow us to define for real
2:03:21
going to allow us to define for real
2:03:21
going to allow us to define for real a middleware where we can execute code
2:03:25
a middleware where we can execute code
2:03:25
a middleware where we can execute code before and after every single request
2:03:27
before and after every single request
2:03:27
before and after every single request happens so we defined a tracer plugin
2:03:30
happens so we defined a tracer plugin
2:03:30
happens so we defined a tracer plugin that is going to work through apn and
2:03:32
that is going to work through apn and
2:03:32
that is going to work through apn and would start a span as soon as a new
2:03:34
would start a span as soon as a new
2:03:34
would start a span as soon as a new request happens and close it once the
2:03:36
request happens and close it once the
2:03:36
request happens and close it once the request is done
2:03:37
request is done
2:03:37
request is done tagging and logging all errors and all
2:03:40
tagging and logging all errors and all
2:03:40
tagging and logging all errors and all sort of useful information
2:03:42
sort of useful information
2:03:42
sort of useful information now what is very interesting about this
2:03:44
now what is very interesting about this
2:03:44
now what is very interesting about this is that if we also trace
2:03:45
is that if we also trace
2:03:46
is that if we also trace or wrapper functions and the sdk
2:03:48
or wrapper functions and the sdk
2:03:48
or wrapper functions and the sdk functions
2:03:49
functions
2:03:49
functions we can finally have a flame graph pair
2:03:52
we can finally have a flame graph pair
2:03:52
we can finally have a flame graph pair operation that is going to show
2:03:54
operation that is going to show
2:03:54
operation that is going to show a timeline telling how much time each
2:03:57
a timeline telling how much time each
2:03:57
a timeline telling how much time each operation took
2:03:58
operation took
2:03:58
operation took in this case we are looking at a
2:04:00
in this case we are looking at a
2:04:00
in this case we are looking at a modified document by id operation there
2:04:03
modified document by id operation there
2:04:03
modified document by id operation there in blue that we will see later but for
2:04:05
in blue that we will see later but for
2:04:05
in blue that we will see later but for now it is
2:04:06
now it is
2:04:06
now it is not by checking that it's doing two
2:04:08
not by checking that it's doing two
2:04:08
not by checking that it's doing two operations under the covers where the
2:04:10
operations under the covers where the
2:04:10
operations under the covers where the request is taking most part of it
2:04:12
request is taking most part of it
2:04:12
request is taking most part of it but not all of it this level of
2:04:15
but not all of it this level of
2:04:15
but not all of it this level of information has been proven to be
2:04:17
information has been proven to be
2:04:17
information has been proven to be of an enormous value since
2:04:20
of an enormous value since
2:04:20
of an enormous value since it is extremely easy to find slow
2:04:22
it is extremely easy to find slow
2:04:22
it is extremely easy to find slow requests and bottlenecks and
2:04:23
requests and bottlenecks and
2:04:24
requests and bottlenecks and errors at this point technically we
2:04:25
errors at this point technically we
2:04:26
errors at this point technically we could go on
2:04:27
could go on
2:04:27
could go on to our previous uh decorators and
2:04:29
to our previous uh decorators and
2:04:29
to our previous uh decorators and replace them
2:04:30
replace them
2:04:30
replace them uh but it's not really like subtracting
2:04:34
uh but it's not really like subtracting
2:04:34
uh but it's not really like subtracting value using these plugins because
2:04:36
value using these plugins because
2:04:36
value using these plugins because there are still some features that we
2:04:38
there are still some features that we
2:04:38
there are still some features that we worked out
2:04:39
worked out
2:04:39
worked out in order to improve performance and
2:04:41
in order to improve performance and
2:04:41
in order to improve performance and reliability for example that are relying
2:04:43
reliability for example that are relying
2:04:43
reliability for example that are relying on the same pattern
2:04:45
on the same pattern
2:04:45
on the same pattern so for performance and reliability we
2:04:47
so for performance and reliability we
2:04:47
so for performance and reliability we followed a few strategies that are
2:04:49
followed a few strategies that are
2:04:49
followed a few strategies that are complementary to each other
2:04:51
complementary to each other
2:04:51
complementary to each other and they are implemented in several
2:04:53
and they are implemented in several
2:04:53
and they are implemented in several different ways
2:04:55
different ways
2:04:55
different ways the first and first one is adding a
2:04:59
the first and first one is adding a
2:04:59
the first and first one is adding a retry
2:04:59
retry
2:04:59
retry and a semaphore layer in order to
2:05:02
and a semaphore layer in order to
2:05:02
and a semaphore layer in order to improve all reliability on every cosmos
2:05:04
improve all reliability on every cosmos
2:05:04
improve all reliability on every cosmos operation
2:05:05
operation
2:05:05
operation as you can see in the snippet they work
2:05:06
as you can see in the snippet they work
2:05:06
as you can see in the snippet they work in the exact same manner as the stat
2:05:09
in the exact same manner as the stat
2:05:09
in the exact same manner as the stat decimeter
2:05:09
decimeter
2:05:09
decimeter and the logger that we have already seen
2:05:12
and the logger that we have already seen
2:05:12
and the logger that we have already seen with retry is going to give
2:05:14
with retry is going to give
2:05:14
with retry is going to give a retry strategy for android sdk
2:05:17
a retry strategy for android sdk
2:05:17
a retry strategy for android sdk requests now since
2:05:18
requests now since
2:05:18
requests now since 2017 the retry logic in the sdk
2:05:21
2017 the retry logic in the sdk
2:05:21
2017 the retry logic in the sdk improved a lot and there is no strict
2:05:23
improved a lot and there is no strict
2:05:23
improved a lot and there is no strict requirement
2:05:25
requirement
2:05:25
requirement to have a custom retry layer for network
2:05:28
to have a custom retry layer for network
2:05:28
to have a custom retry layer for network errors anymore
2:05:29
errors anymore
2:05:29
errors anymore but there are still some cases where the
2:05:31
but there are still some cases where the
2:05:31
but there are still some cases where the sdk request can fail
2:05:33
sdk request can fail
2:05:33
sdk request can fail but still be retriable for example when
2:05:35
but still be retriable for example when
2:05:35
but still be retriable for example when the right session is not available this
2:05:37
the right session is not available this
2:05:38
the right session is not available this has been very useful
2:05:39
has been very useful
2:05:39
has been very useful for such cases we also have
2:05:42
for such cases we also have
2:05:42
for such cases we also have a layer that adds a semaphore in front
2:05:45
a layer that adds a semaphore in front
2:05:45
a layer that adds a semaphore in front of all of these requests and this is
2:05:46
of all of these requests and this is
2:05:46
of all of these requests and this is going to help us making sure that we
2:05:48
going to help us making sure that we
2:05:48
going to help us making sure that we don't take the event loop it helps
2:05:51
don't take the event loop it helps
2:05:51
don't take the event loop it helps it helps controlling the concurrency uh
2:05:54
it helps controlling the concurrency uh
2:05:54
it helps controlling the concurrency uh how many operations are going to be
2:05:55
how many operations are going to be
2:05:56
how many operations are going to be running at the exact same time if we do
2:05:57
running at the exact same time if we do
2:05:57
running at the exact same time if we do for example a promise all with thousands
2:05:59
for example a promise all with thousands
2:05:59
for example a promise all with thousands of
2:06:00
of
2:06:00
of operations at the same time we will make
2:06:02
operations at the same time we will make
2:06:02
operations at the same time we will make sure that we are not taking the event
2:06:03
sure that we are not taking the event
2:06:03
sure that we are not taking the event loop
2:06:04
loop
2:06:04
loop and we are not running a lot of
2:06:06
and we are not running a lot of
2:06:06
and we are not running a lot of operations in the exact same millisecond
2:06:09
operations in the exact same millisecond
2:06:09
operations in the exact same millisecond with this we complete all of the
2:06:10
with this we complete all of the
2:06:10
with this we complete all of the decorators that we have in front of
2:06:12
decorators that we have in front of
2:06:12
decorators that we have in front of every single operation
2:06:13
every single operation
2:06:14
every single operation but there are more interesting things
2:06:15
but there are more interesting things
2:06:15
but there are more interesting things that we did to improve performance and
2:06:18
that we did to improve performance and
2:06:18
that we did to improve performance and reliability
2:06:20
reliability
2:06:20
reliability um in our experience when we are reading
2:06:22
um in our experience when we are reading
2:06:22
um in our experience when we are reading a document and that is taking more than
2:06:25
a document and that is taking more than
2:06:25
a document and that is taking more than half a second
2:06:26
half a second
2:06:26
half a second it usually times out this has to do with
2:06:28
it usually times out this has to do with
2:06:28
it usually times out this has to do with the cosmos gateway
2:06:29
the cosmos gateway
2:06:30
the cosmos gateway being too pc and usually a simple retry
2:06:32
being too pc and usually a simple retry
2:06:32
being too pc and usually a simple retry is going to help in this case
2:06:34
is going to help in this case
2:06:34
is going to help in this case we have seen it so in order to mobility
2:06:36
we have seen it so in order to mobility
2:06:36
we have seen it so in order to mobility pipeline many cases
2:06:38
pipeline many cases
2:06:38
pipeline many cases where a request took exactly the time
2:06:41
where a request took exactly the time
2:06:41
where a request took exactly the time out
2:06:42
out
2:06:42
out that we have set plus one second back
2:06:44
that we have set plus one second back
2:06:44
that we have set plus one second back off plus a few milliseconds and this is
2:06:46
off plus a few milliseconds and this is
2:06:46
off plus a few milliseconds and this is precisely because the very first retry
2:06:48
precisely because the very first retry
2:06:48
precisely because the very first retry is going through correctly so a good
2:06:52
is going through correctly so a good
2:06:52
is going through correctly so a good strategy to overcome this issue
2:06:54
strategy to overcome this issue
2:06:54
strategy to overcome this issue is to have an early timeout strategy for
2:06:56
is to have an early timeout strategy for
2:06:56
is to have an early timeout strategy for the first try
2:06:57
the first try
2:06:57
the first try that timeout can be increasing with the
2:07:00
that timeout can be increasing with the
2:07:00
that timeout can be increasing with the next three tries but it is very
2:07:02
next three tries but it is very
2:07:02
next three tries but it is very important to know that it has to be tied
2:07:04
important to know that it has to be tied
2:07:04
important to know that it has to be tied to
2:07:05
to
2:07:05
to each operation for example reading a
2:07:07
each operation for example reading a
2:07:07
each operation for example reading a document is not going to take the same
2:07:09
document is not going to take the same
2:07:09
document is not going to take the same amount of time
2:07:10
amount of time
2:07:10
amount of time as a query so we can be that optimistic
2:07:13
as a query so we can be that optimistic
2:07:13
as a query so we can be that optimistic it is for this that we defined a plugin
2:07:15
it is for this that we defined a plugin
2:07:15
it is for this that we defined a plugin in a similar way that
2:07:17
in a similar way that
2:07:17
in a similar way that we have seen for the tracer that allows
2:07:19
we have seen for the tracer that allows
2:07:19
we have seen for the tracer that allows to pass
2:07:20
to pass
2:07:20
to pass a times out policy to a wrapper where we
2:07:23
a times out policy to a wrapper where we
2:07:23
a times out policy to a wrapper where we can
2:07:24
can
2:07:24
can specify not only the timeouts that we
2:07:26
specify not only the timeouts that we
2:07:26
specify not only the timeouts that we want to get in a sequence but also the
2:07:28
want to get in a sequence but also the
2:07:28
want to get in a sequence but also the amount of free trials and the backup
2:07:30
amount of free trials and the backup
2:07:30
amount of free trials and the backup strategy
2:07:31
strategy
2:07:31
strategy for each of those this has been very
2:07:34
for each of those this has been very
2:07:34
for each of those this has been very very useful for us
2:07:35
very useful for us
2:07:35
very useful for us and it helps reduce dramatically the max
2:07:37
and it helps reduce dramatically the max
2:07:37
and it helps reduce dramatically the max time on the p99
2:07:39
time on the p99
2:07:39
time on the p99 in our apis
2:07:42
another one that is one of my favorites
2:07:44
another one that is one of my favorites
2:07:44
another one that is one of my favorites when it comes to reliability or system
2:07:46
when it comes to reliability or system
2:07:46
when it comes to reliability or system is the ability to perform
2:07:48
is the ability to perform
2:07:48
is the ability to perform safe updates avoiding race conditions
2:07:51
safe updates avoiding race conditions
2:07:51
safe updates avoiding race conditions and potential data loss
2:07:53
and potential data loss
2:07:53
and potential data loss in systems where we are processing like
2:07:55
in systems where we are processing like
2:07:56
in systems where we are processing like in hours
2:07:57
in hours
2:07:57
in hours thousands of requests per second and we
2:07:59
thousands of requests per second and we
2:07:59
thousands of requests per second and we have spawned jobs that are processing
2:08:01
have spawned jobs that are processing
2:08:01
have spawned jobs that are processing and accessing and mutating the same data
2:08:03
and accessing and mutating the same data
2:08:04
and accessing and mutating the same data concurrently there is a very high risk
2:08:06
concurrently there is a very high risk
2:08:06
concurrently there is a very high risk of overriding data because of race
2:08:09
of overriding data because of race
2:08:09
of overriding data because of race conditions
2:08:10
conditions
2:08:10
conditions say that you have this scenario where
2:08:11
say that you have this scenario where
2:08:11
say that you have this scenario where you are reading a document
2:08:13
you are reading a document
2:08:13
you are reading a document and then you update the document locally
2:08:15
and then you update the document locally
2:08:15
and then you update the document locally and then you push the mutation to the
2:08:17
and then you push the mutation to the
2:08:17
and then you push the mutation to the server
2:08:17
server
2:08:17
server the chances that the item was modified
2:08:19
the chances that the item was modified
2:08:20
the chances that the item was modified by the time you push
2:08:21
by the time you push
2:08:21
by the time you push are very high and can lead to data loss
2:08:24
are very high and can lead to data loss
2:08:24
are very high and can lead to data loss because you are updating
2:08:25
because you are updating
2:08:25
because you are updating stale data into your system rest api
2:08:28
stale data into your system rest api
2:08:28
stale data into your system rest api solve this with
2:08:29
solve this with
2:08:29
solve this with percent resources using e-tax and cosmos
2:08:32
percent resources using e-tax and cosmos
2:08:32
percent resources using e-tax and cosmos works exactly in this way
2:08:34
works exactly in this way
2:08:34
works exactly in this way so we wanted to bring this behavior to
2:08:36
so we wanted to bring this behavior to
2:08:36
so we wanted to bring this behavior to every single a
2:08:37
every single a
2:08:37
every single a api every single place where we were
2:08:40
api every single place where we were
2:08:40
api every single place where we were pushing changes
2:08:41
pushing changes
2:08:41
pushing changes uh for a document and
2:08:45
uh for a document and
2:08:45
uh for a document and we try in case we got an error with the
2:08:48
we try in case we got an error with the
2:08:48
we try in case we got an error with the etag
2:08:49
etag
2:08:49
etag so we modeled this high
2:08:52
so we modeled this high
2:08:52
so we modeled this high level tool modify document by id that
2:08:55
level tool modify document by id that
2:08:55
level tool modify document by id that would help us
2:08:56
would help us
2:08:56
would help us in those scenarios it works just by
2:08:58
in those scenarios it works just by
2:08:58
in those scenarios it works just by receiving the collection name
2:09:00
receiving the collection name
2:09:00
receiving the collection name the document id and an updater function
2:09:03
the document id and an updater function
2:09:04
the document id and an updater function so what it's going to do is it's going
2:09:05
so what it's going to do is it's going
2:09:05
so what it's going to do is it's going to read the document by the id or the
2:09:07
to read the document by the id or the
2:09:07
to read the document by the id or the information that is provided and then
2:09:09
information that is provided and then
2:09:09
information that is provided and then it's going to apply
2:09:10
it's going to apply
2:09:10
it's going to apply that modifier updater function if it
2:09:13
that modifier updater function if it
2:09:13
that modifier updater function if it performed
2:09:14
performed
2:09:14
performed any changes it's going to push the
2:09:17
any changes it's going to push the
2:09:17
any changes it's going to push the the change and in case it fails because
2:09:19
the change and in case it fails because
2:09:20
the change and in case it fails because of the e-tag or the resource version
2:09:21
of the e-tag or the resource version
2:09:22
of the e-tag or the resource version then it's going to retry and apply the
2:09:24
then it's going to retry and apply the
2:09:24
then it's going to retry and apply the function again
2:09:25
function again
2:09:25
function again this was really really helpful for a use
2:09:27
this was really really helpful for a use
2:09:27
this was really really helpful for a use case and it helped us
2:09:29
case and it helped us
2:09:29
case and it helped us to be super sure that we are not
2:09:31
to be super sure that we are not
2:09:31
to be super sure that we are not committing any
2:09:32
committing any
2:09:32
committing any we are not getting into this trouble of
2:09:34
we are not getting into this trouble of
2:09:34
we are not getting into this trouble of overriding data
2:09:36
overriding data
2:09:36
overriding data with stale data
2:09:40
with stale data
2:09:40
with stale data so now let's see how we enrich our
2:09:42
so now let's see how we enrich our
2:09:42
so now let's see how we enrich our models and brought
2:09:43
models and brought
2:09:43
models and brought a declarative way to express what our
2:09:45
a declarative way to express what our
2:09:45
a declarative way to express what our collections look like
2:09:47
collections look like
2:09:47
collections look like in this section we're going to see how
2:09:48
in this section we're going to see how
2:09:48
in this section we're going to see how we define features that are present in
2:09:50
we define features that are present in
2:09:50
we define features that are present in other databases and are essential to
2:09:52
other databases and are essential to
2:09:52
other databases and are essential to operate cosmos in production in my
2:09:54
operate cosmos in production in my
2:09:54
operate cosmos in production in my opinion
2:09:55
opinion
2:09:56
opinion the first and most important of those is
2:09:58
the first and most important of those is
2:09:58
the first and most important of those is time stamps there are many databases out
2:10:00
time stamps there are many databases out
2:10:00
time stamps there are many databases out there that have out of the box support
2:10:02
there that have out of the box support
2:10:02
there that have out of the box support for timestamps those would
2:10:03
for timestamps those would
2:10:03
for timestamps those would those are going to tell you when a
2:10:05
those are going to tell you when a
2:10:05
those are going to tell you when a transaction took place when a document
2:10:06
transaction took place when a document
2:10:06
transaction took place when a document was created when it was last updated
2:10:09
was created when it was last updated
2:10:09
was created when it was last updated and in some cases when it was deleted so
2:10:12
and in some cases when it was deleted so
2:10:12
and in some cases when it was deleted so since we have this sdk wrapper that runs
2:10:15
since we have this sdk wrapper that runs
2:10:15
since we have this sdk wrapper that runs on top of
2:10:16
on top of
2:10:16
on top of every single mutation and operation
2:10:18
every single mutation and operation
2:10:18
every single mutation and operation instead of going to every single place
2:10:20
instead of going to every single place
2:10:20
instead of going to every single place and add these
2:10:21
and add these
2:10:21
and add these fields to different documents we can
2:10:24
fields to different documents we can
2:10:24
fields to different documents we can just put it in our wrapper
2:10:26
just put it in our wrapper
2:10:26
just put it in our wrapper and it is worth mentioning that this is
2:10:28
and it is worth mentioning that this is
2:10:28
and it is worth mentioning that this is very useful for sorting
2:10:30
very useful for sorting
2:10:30
very useful for sorting informations to 13 records um
2:10:33
informations to 13 records um
2:10:33
informations to 13 records um depending on when they were updated and
2:10:35
depending on when they were updated and
2:10:35
depending on when they were updated and cosmos includes already an attribute for
2:10:37
cosmos includes already an attribute for
2:10:37
cosmos includes already an attribute for this
2:10:37
this
2:10:37
this but you can opt out from updating that
2:10:40
but you can opt out from updating that
2:10:40
but you can opt out from updating that attribute when you
2:10:41
attribute when you
2:10:41
attribute when you push an update to a document and we
2:10:43
push an update to a document and we
2:10:43
push an update to a document and we wanted to have
2:10:44
wanted to have
2:10:44
wanted to have that feature which is why we implemented
2:10:47
that feature which is why we implemented
2:10:47
that feature which is why we implemented this
2:10:47
this
2:10:47
this another one that is very closely related
2:10:49
another one that is very closely related
2:10:49
another one that is very closely related to this is soft deletion
2:10:52
to this is soft deletion
2:10:52
to this is soft deletion um we never remove data for real what we
2:10:56
um we never remove data for real what we
2:10:56
um we never remove data for real what we do instead is that we store a timestamp
2:10:58
do instead is that we store a timestamp
2:10:58
do instead is that we store a timestamp that is going to tell us when an item
2:11:00
that is going to tell us when an item
2:11:00
that is going to tell us when an item was deleted
2:11:04
the reasons why we have this is actually
2:11:06
the reasons why we have this is actually
2:11:06
the reasons why we have this is actually pretty obvious
2:11:07
pretty obvious
2:11:07
pretty obvious [Applause]
2:11:08
[Applause]
2:11:08
[Applause] removing data for real is very risky and
2:11:12
removing data for real is very risky and
2:11:12
removing data for real is very risky and in case that you are rolling some
2:11:14
in case that you are rolling some
2:11:14
in case that you are rolling some regression or some baggy code
2:11:16
regression or some baggy code
2:11:16
regression or some baggy code you don't want to just remove data in an
2:11:19
you don't want to just remove data in an
2:11:19
you don't want to just remove data in an unrecordable
2:11:20
unrecordable
2:11:20
unrecordable unrecoverable manner in the snippets you
2:11:23
unrecoverable manner in the snippets you
2:11:23
unrecoverable manner in the snippets you can see
2:11:24
can see
2:11:24
can see uh how we usually remove documents we
2:11:27
uh how we usually remove documents we
2:11:27
uh how we usually remove documents we have set a timestamp through
2:11:28
have set a timestamp through
2:11:28
have set a timestamp through a wrapper function and from that point
2:11:31
a wrapper function and from that point
2:11:31
a wrapper function and from that point on we are going to ignore those
2:11:33
on we are going to ignore those
2:11:33
on we are going to ignore those documents
2:11:34
documents
2:11:34
documents of course this comes with a cost because
2:11:35
of course this comes with a cost because
2:11:36
of course this comes with a cost because every single query is going to have to
2:11:37
every single query is going to have to
2:11:37
every single query is going to have to include automatically a predicate
2:11:40
include automatically a predicate
2:11:40
include automatically a predicate to leave all of these records outside of
2:11:42
to leave all of these records outside of
2:11:42
to leave all of these records outside of every
2:11:43
every
2:11:43
every query of course this behavior can be opt
2:11:46
query of course this behavior can be opt
2:11:46
query of course this behavior can be opt out
2:11:48
and this doesn't mean that we are
2:11:50
and this doesn't mean that we are
2:11:50
and this doesn't mean that we are keeping records forever and ever around
2:11:52
keeping records forever and ever around
2:11:52
keeping records forever and ever around we have some planning processes that are
2:11:55
we have some planning processes that are
2:11:55
we have some planning processes that are going to be removing data
2:11:56
going to be removing data
2:11:56
going to be removing data after a safe period of time
2:12:00
after a safe period of time
2:12:00
after a safe period of time depending on the entities that are going
2:12:02
depending on the entities that are going
2:12:02
depending on the entities that are going to be
2:12:03
to be
2:12:03
to be cleaned up so
2:12:07
cleaned up so
2:12:07
cleaned up so another thing that we didn't mention
2:12:08
another thing that we didn't mention
2:12:08
another thing that we didn't mention just yet is partitioning
2:12:10
just yet is partitioning
2:12:10
just yet is partitioning in cosmos is very important to partition
2:12:12
in cosmos is very important to partition
2:12:12
in cosmos is very important to partition collections correctly
2:12:14
collections correctly
2:12:14
collections correctly and it is something that obviously we
2:12:16
and it is something that obviously we
2:12:16
and it is something that obviously we didn't do at the beginning
2:12:18
didn't do at the beginning
2:12:18
didn't do at the beginning in our wrapper we automatically use the
2:12:20
in our wrapper we automatically use the
2:12:20
in our wrapper we automatically use the id as the partition
2:12:21
id as the partition
2:12:21
id as the partition key because all of our collections were
2:12:24
key because all of our collections were
2:12:24
key because all of our collections were partitioned by id
2:12:26
partitioned by id
2:12:26
partitioned by id so when the time came that we have to
2:12:29
so when the time came that we have to
2:12:29
so when the time came that we have to use a different partition key the
2:12:30
use a different partition key the
2:12:30
use a different partition key the wrapper just didn't know how to
2:12:32
wrapper just didn't know how to
2:12:32
wrapper just didn't know how to inject the right partition key so we
2:12:34
inject the right partition key so we
2:12:34
inject the right partition key so we decided to create a very simple
2:12:36
decided to create a very simple
2:12:36
decided to create a very simple configuration object
2:12:37
configuration object
2:12:38
configuration object that would enumerate all of the
2:12:39
that would enumerate all of the
2:12:39
that would enumerate all of the collections that our system has
2:12:41
collections that our system has
2:12:41
collections that our system has and what their partition ids were this
2:12:44
and what their partition ids were this
2:12:44
and what their partition ids were this enabled us to do
2:12:45
enabled us to do
2:12:45
enabled us to do very interesting things aside from
2:12:49
very interesting things aside from
2:12:49
very interesting things aside from injecting the partition keys
2:12:51
injecting the partition keys
2:12:51
injecting the partition keys automatically in the snippets you can
2:12:52
automatically in the snippets you can
2:12:52
automatically in the snippets you can see
2:12:53
see
2:12:53
see how we are just enumerating collections
2:12:55
how we are just enumerating collections
2:12:55
how we are just enumerating collections we are going to be referencing later
2:12:57
we are going to be referencing later
2:12:57
we are going to be referencing later those collections rather than hard
2:12:58
those collections rather than hard
2:12:58
those collections rather than hard coding the strings
2:13:00
coding the strings
2:13:00
coding the strings and we can define their partitions
2:13:04
and we can define their partitions
2:13:04
and we can define their partitions on the case where we have a
2:13:06
on the case where we have a
2:13:06
on the case where we have a configuration object like this
2:13:09
configuration object like this
2:13:09
configuration object like this and we are running a cross-partition
2:13:10
and we are running a cross-partition
2:13:10
and we are running a cross-partition query through a helper
2:13:12
query through a helper
2:13:12
query through a helper what is going to happen is that we can
2:13:13
what is going to happen is that we can
2:13:14
what is going to happen is that we can analyze the query and we can see
2:13:16
analyze the query and we can see
2:13:16
analyze the query and we can see if the partition key is actually being
2:13:18
if the partition key is actually being
2:13:18
if the partition key is actually being included in the query
2:13:20
included in the query
2:13:20
included in the query with the sdk version 3
2:13:23
with the sdk version 3
2:13:23
with the sdk version 3 um if the partition key is included in
2:13:26
um if the partition key is included in
2:13:26
um if the partition key is included in the query
2:13:27
the query
2:13:27
the query it's going to work in a very efficient
2:13:28
it's going to work in a very efficient
2:13:28
it's going to work in a very efficient way but if it's not it's going to be
2:13:30
way but if it's not it's going to be
2:13:30
way but if it's not it's going to be doing a cross-partition query
2:13:32
doing a cross-partition query
2:13:32
doing a cross-partition query and that is going to be a very high a
2:13:36
and that is going to be a very high a
2:13:36
and that is going to be a very high a very costly operation
2:13:37
very costly operation
2:13:37
very costly operation so we wanted to make uh developers very
2:13:40
so we wanted to make uh developers very
2:13:40
so we wanted to make uh developers very aware
2:13:41
aware
2:13:41
aware of this fact and force everybody to
2:13:44
of this fact and force everybody to
2:13:44
of this fact and force everybody to into the awareness of of of making
2:13:47
into the awareness of of of making
2:13:47
into the awareness of of of making cross-partition queries
2:13:48
cross-partition queries
2:13:48
cross-partition queries so we have managed to use this
2:13:51
so we have managed to use this
2:13:51
so we have managed to use this configuration
2:13:53
configuration
2:13:53
configuration in a way that it throws uh asynchronous
2:13:55
in a way that it throws uh asynchronous
2:13:55
in a way that it throws uh asynchronous error
2:13:56
error
2:13:56
error if you are running into a
2:13:57
if you are running into a
2:13:58
if you are running into a cross-production query without
2:13:59
cross-production query without
2:13:59
cross-production query without specifically telling so through a flag
2:14:03
specifically telling so through a flag
2:14:03
specifically telling so through a flag this helps us save a lot of
2:14:05
this helps us save a lot of
2:14:05
this helps us save a lot of cross-partition queries
2:14:07
cross-partition queries
2:14:07
cross-partition queries um and every related to this
2:14:10
um and every related to this
2:14:10
um and every related to this configuration everything
2:14:11
configuration everything
2:14:11
configuration everything fall into place when we started building
2:14:14
fall into place when we started building
2:14:14
fall into place when we started building a testing framework
2:14:15
a testing framework
2:14:15
a testing framework uh that would use our in-memory replica
2:14:18
uh that would use our in-memory replica
2:14:18
uh that would use our in-memory replica of the cosmos tv server
2:14:20
of the cosmos tv server
2:14:20
of the cosmos tv server that link that you're seeing there is uh
2:14:22
that link that you're seeing there is uh
2:14:22
that link that you're seeing there is uh public and you can access and use it
2:14:24
public and you can access and use it
2:14:24
public and you can access and use it today it is a cosmo ct server that works
2:14:27
today it is a cosmo ct server that works
2:14:27
today it is a cosmo ct server that works in memory and it has pretty much the
2:14:28
in memory and it has pretty much the
2:14:28
in memory and it has pretty much the same api
2:14:29
same api
2:14:29
same api as the one you use in production so in
2:14:32
as the one you use in production so in
2:14:32
as the one you use in production so in order to set up
2:14:33
order to set up
2:14:33
order to set up this server we obviously need to create
2:14:36
this server we obviously need to create
2:14:36
this server we obviously need to create containers
2:14:37
containers
2:14:37
containers the create user defined function
2:14:39
the create user defined function
2:14:39
the create user defined function composite indexes and so on
2:14:41
composite indexes and so on
2:14:41
composite indexes and so on because we already had a configuration
2:14:42
because we already had a configuration
2:14:42
because we already had a configuration for the wrapper with all of this
2:14:44
for the wrapper with all of this
2:14:44
for the wrapper with all of this information we just needed to enrich it
2:14:46
information we just needed to enrich it
2:14:46
information we just needed to enrich it a little bit in order to generate the
2:14:50
a little bit in order to generate the
2:14:50
a little bit in order to generate the database server that is suitable for our
2:14:53
database server that is suitable for our
2:14:53
database server that is suitable for our schema
2:14:55
schema
2:14:55
schema and even though those properties like
2:14:57
and even though those properties like
2:14:58
and even though those properties like you need keys
2:14:58
you need keys
2:14:58
you need keys and composite indexes are not really
2:15:01
and composite indexes are not really
2:15:01
and composite indexes are not really used inside of the wrapper it becomes
2:15:02
used inside of the wrapper it becomes
2:15:02
used inside of the wrapper it becomes very handy to have them around because
2:15:04
very handy to have them around because
2:15:04
very handy to have them around because you can just have a look at your
2:15:06
you can just have a look at your
2:15:06
you can just have a look at your configuration and have a very
2:15:08
configuration and have a very
2:15:08
configuration and have a very high level view of what the schema looks
2:15:12
high level view of what the schema looks
2:15:12
high level view of what the schema looks like
2:15:12
like
2:15:12
like and that of respecting the schema free
2:15:15
and that of respecting the schema free
2:15:15
and that of respecting the schema free and flexibility
2:15:16
and flexibility
2:15:16
and flexibility of the cosmos database
2:15:19
of the cosmos database
2:15:19
of the cosmos database so this was an incredible useful future
2:15:22
so this was an incredible useful future
2:15:22
so this was an incredible useful future for us
2:15:23
for us
2:15:23
for us in our testing pipeline we have this
2:15:25
in our testing pipeline we have this
2:15:25
in our testing pipeline we have this in-memory server that boots
2:15:27
in-memory server that boots
2:15:27
in-memory server that boots clears and then self-configures
2:15:28
clears and then self-configures
2:15:28
clears and then self-configures automatically for every single test
2:15:30
automatically for every single test
2:15:30
automatically for every single test from this point in time testing became a
2:15:33
from this point in time testing became a
2:15:33
from this point in time testing became a matter of creating a bunch of documents
2:15:34
matter of creating a bunch of documents
2:15:34
matter of creating a bunch of documents to reproduce in a scenario
2:15:36
to reproduce in a scenario
2:15:36
to reproduce in a scenario run the code and then run the assertions
2:15:38
run the code and then run the assertions
2:15:38
run the code and then run the assertions and we didn't have to think about
2:15:39
and we didn't have to think about
2:15:39
and we didn't have to think about setting a server
2:15:40
setting a server
2:15:40
setting a server anymore as a matter of fact this gave us
2:15:43
anymore as a matter of fact this gave us
2:15:43
anymore as a matter of fact this gave us the idea of a feature that we called
2:15:45
the idea of a feature that we called
2:15:45
the idea of a feature that we called database snapshot testing
2:15:48
database snapshot testing
2:15:48
database snapshot testing so based on the configuration it would
2:15:51
so based on the configuration it would
2:15:51
so based on the configuration it would be very very easy to have a function
2:15:53
be very very easy to have a function
2:15:53
be very very easy to have a function that we call
2:15:53
that we call
2:15:54
that we call that would go through every single uh
2:15:56
that would go through every single uh
2:15:56
that would go through every single uh collection
2:15:58
collection
2:15:58
collection querying all of its information and
2:16:00
querying all of its information and
2:16:00
querying all of its information and creating a sorted
2:16:01
creating a sorted
2:16:01
creating a sorted object that you can use to retrieve the
2:16:04
object that you can use to retrieve the
2:16:04
object that you can use to retrieve the whole state
2:16:05
whole state
2:16:05
whole state of the in-memory database so that way
2:16:08
of the in-memory database so that way
2:16:08
of the in-memory database so that way testing becomes a matter of preparing
2:16:11
testing becomes a matter of preparing
2:16:11
testing becomes a matter of preparing the
2:16:11
the
2:16:11
the environment doing a get on this
2:16:15
environment doing a get on this
2:16:15
environment doing a get on this whole database state and then running
2:16:19
whole database state and then running
2:16:19
whole database state and then running the effects getting the state again
2:16:21
the effects getting the state again
2:16:21
the effects getting the state again differing
2:16:22
differing
2:16:22
differing and snapshotting that everything is
2:16:24
and snapshotting that everything is
2:16:24
and snapshotting that everything is working as expected
2:16:26
working as expected
2:16:26
working as expected it's a very handy way to test side
2:16:28
it's a very handy way to test side
2:16:28
it's a very handy way to test side effects
2:16:30
effects
2:16:30
effects um yeah
2:16:33
um yeah
2:16:33
um yeah so the next important feature that i
2:16:35
so the next important feature that i
2:16:35
so the next important feature that i want to talk about
2:16:36
want to talk about
2:16:36
want to talk about is that what we call secondary keys and
2:16:38
is that what we call secondary keys and
2:16:38
is that what we call secondary keys and this is one of the most
2:16:39
this is one of the most
2:16:39
this is one of the most important uh that we have faced in our
2:16:43
important uh that we have faced in our
2:16:43
important uh that we have faced in our journey with cosmos
2:16:45
journey with cosmos
2:16:45
journey with cosmos this was crucial for us to minimize the
2:16:47
this was crucial for us to minimize the
2:16:47
this was crucial for us to minimize the amount of cross-partition queries and to
2:16:49
amount of cross-partition queries and to
2:16:49
amount of cross-partition queries and to ensure unique values across different
2:16:50
ensure unique values across different
2:16:50
ensure unique values across different collections and partitions
2:16:53
collections and partitions
2:16:53
collections and partitions say that you have a model like this with
2:16:55
say that you have a model like this with
2:16:55
say that you have a model like this with users and teams
2:16:57
users and teams
2:16:57
users and teams in the user is going to hold some data
2:17:02
in the user is going to hold some data
2:17:02
in the user is going to hold some data that in a relational database would be
2:17:05
that in a relational database would be
2:17:05
that in a relational database would be primary keys
2:17:05
primary keys
2:17:06
primary keys like values that are unique to the
2:17:08
like values that are unique to the
2:17:08
like values that are unique to the system and can be used to index the
2:17:10
system and can be used to index the
2:17:10
system and can be used to index the entity
2:17:11
entity
2:17:11
entity on the other hand you have a team that
2:17:13
on the other hand you have a team that
2:17:13
on the other hand you have a team that also has a couple of keys like the id
2:17:16
also has a couple of keys like the id
2:17:16
also has a couple of keys like the id and the stock now what is interesting
2:17:19
and the stock now what is interesting
2:17:19
and the stock now what is interesting about this use case
2:17:20
about this use case
2:17:20
about this use case is that we want the user username and
2:17:23
is that we want the user username and
2:17:23
is that we want the user username and the
2:17:23
the
2:17:24
the teams log to be in a shared namespace we
2:17:27
teams log to be in a shared namespace we
2:17:27
teams log to be in a shared namespace we want them to be
2:17:28
want them to be
2:17:28
want them to be unique even if they are documents
2:17:31
unique even if they are documents
2:17:31
unique even if they are documents stored in different collections
2:17:35
stored in different collections
2:17:35
stored in different collections besides that we need to have an optimal
2:17:37
besides that we need to have an optimal
2:17:37
besides that we need to have an optimal access
2:17:38
access
2:17:38
access to users and teams using
2:17:42
to users and teams using
2:17:42
to users and teams using their unique values when we choose the
2:17:45
their unique values when we choose the
2:17:45
their unique values when we choose the partition key
2:17:46
partition key
2:17:46
partition key we would say that it is the id in these
2:17:49
we would say that it is the id in these
2:17:49
we would say that it is the id in these cases because it's the most common
2:17:51
cases because it's the most common
2:17:51
cases because it's the most common access pattern
2:17:52
access pattern
2:17:52
access pattern but still we're going to have to have
2:17:53
but still we're going to have to have
2:17:53
but still we're going to have to have many places where we need to find a user
2:17:55
many places where we need to find a user
2:17:55
many places where we need to find a user by email for example and that's going to
2:17:57
by email for example and that's going to
2:17:57
by email for example and that's going to be a cross-partition query that over
2:17:59
be a cross-partition query that over
2:17:59
be a cross-partition query that over time
2:17:59
time
2:17:59
time can be very very damaging so how do we
2:18:02
can be very very damaging so how do we
2:18:02
can be very very damaging so how do we solve this thing
2:18:05
solve this thing
2:18:05
solve this thing this is why i said that the
2:18:06
this is why i said that the
2:18:06
this is why i said that the configuration was like the cornerstone
2:18:09
configuration was like the cornerstone
2:18:09
configuration was like the cornerstone of this advanced tooling we just
2:18:12
of this advanced tooling we just
2:18:12
of this advanced tooling we just added some configuration values that
2:18:15
added some configuration values that
2:18:15
added some configuration values that would tell what are the secondary keys
2:18:17
would tell what are the secondary keys
2:18:17
would tell what are the secondary keys in our system for example for users we
2:18:20
in our system for example for users we
2:18:20
in our system for example for users we will say that the key
2:18:21
will say that the key
2:18:21
will say that the key email and the key username are going to
2:18:23
email and the key username are going to
2:18:23
email and the key username are going to hold secondary keys
2:18:25
hold secondary keys
2:18:25
hold secondary keys and we will say what is the collection
2:18:27
and we will say what is the collection
2:18:27
and we will say what is the collection where we want to store
2:18:29
where we want to store
2:18:29
where we want to store what is going to represent the key then
2:18:32
what is going to represent the key then
2:18:32
what is going to represent the key then optionally you can also give a namespace
2:18:35
optionally you can also give a namespace
2:18:35
optionally you can also give a namespace to make sure that you can have leaving
2:18:37
to make sure that you can have leaving
2:18:37
to make sure that you can have leaving keys
2:18:38
keys
2:18:38
keys in the same name spaces that would
2:18:40
in the same name spaces that would
2:18:40
in the same name spaces that would collide with each other
2:18:41
collide with each other
2:18:41
collide with each other to ensure uniqueness across usernames
2:18:44
to ensure uniqueness across usernames
2:18:44
to ensure uniqueness across usernames usernames
2:18:45
usernames
2:18:45
usernames and slots in that case
2:18:48
and slots in that case
2:18:48
and slots in that case of course when we
2:18:51
of course when we
2:18:51
of course when we run now uh create document the downside
2:18:54
run now uh create document the downside
2:18:54
run now uh create document the downside is that we are going to be creating not
2:18:55
is that we are going to be creating not
2:18:55
is that we are going to be creating not only one document but also documents for
2:18:57
only one document but also documents for
2:18:57
only one document but also documents for the keys
2:18:58
the keys
2:18:58
the keys themselves um that key those keys are
2:19:01
themselves um that key those keys are
2:19:01
themselves um that key those keys are going to
2:19:01
going to
2:19:01
going to keep information about the original
2:19:03
keep information about the original
2:19:03
keep information about the original collection the renault partition key the
2:19:05
collection the renault partition key the
2:19:05
collection the renault partition key the original value
2:19:07
original value
2:19:07
original value etc so what is very
2:19:11
etc so what is very
2:19:11
etc so what is very what is really great about this thing is
2:19:13
what is really great about this thing is
2:19:13
what is really great about this thing is not only that it is allowing
2:19:15
not only that it is allowing
2:19:15
not only that it is allowing a uniqueness but also we need
2:19:18
a uniqueness but also we need
2:19:18
a uniqueness but also we need when we get document we can have a
2:19:20
when we get document we can have a
2:19:20
when we get document we can have a helper that is going to get documents
2:19:21
helper that is going to get documents
2:19:22
helper that is going to get documents by keys or no you can simply tell what
2:19:24
by keys or no you can simply tell what
2:19:24
by keys or no you can simply tell what is the
2:19:25
is the
2:19:25
is the collection that you are accessing what
2:19:27
collection that you are accessing what
2:19:27
collection that you are accessing what is the key that you want to access and
2:19:29
is the key that you want to access and
2:19:29
is the key that you want to access and what is the unique value
2:19:30
what is the unique value
2:19:30
what is the unique value and instead of doing a cross-partition
2:19:32
and instead of doing a cross-partition
2:19:32
and instead of doing a cross-partition query to retrieve that value
2:19:34
query to retrieve that value
2:19:34
query to retrieve that value we are going to do two consecutive get
2:19:36
we are going to do two consecutive get
2:19:36
we are going to do two consecutive get documents by id
2:19:37
documents by id
2:19:37
documents by id that can be up to 50 milliseconds which
2:19:40
that can be up to 50 milliseconds which
2:19:40
that can be up to 50 milliseconds which is much much much better
2:19:42
is much much much better
2:19:42
is much much much better than a cross partition query of course
2:19:44
than a cross partition query of course
2:19:44
than a cross partition query of course there are still two reads
2:19:46
there are still two reads
2:19:46
there are still two reads but i think that the balance is pretty
2:19:48
but i think that the balance is pretty
2:19:48
but i think that the balance is pretty good overall
2:19:49
good overall
2:19:49
good overall the most challenging part about this is
2:19:51
the most challenging part about this is
2:19:51
the most challenging part about this is that we have to model
2:19:53
that we have to model
2:19:53
that we have to model sort of transactions we don't have
2:19:54
sort of transactions we don't have
2:19:54
sort of transactions we don't have transactions like we mentioned
2:19:57
transactions like we mentioned
2:19:57
transactions like we mentioned on this morning so we had to be very
2:19:59
on this morning so we had to be very
2:19:59
on this morning so we had to be very careful when modeling this to make sure
2:20:01
careful when modeling this to make sure
2:20:02
careful when modeling this to make sure that if there is a failure we could roll
2:20:04
that if there is a failure we could roll
2:20:04
that if there is a failure we could roll back the changes that we already done
2:20:06
back the changes that we already done
2:20:06
back the changes that we already done and we don't leave stale data still we
2:20:08
and we don't leave stale data still we
2:20:08
and we don't leave stale data still we let some other tooling in place
2:20:10
let some other tooling in place
2:20:10
let some other tooling in place for re-indexation and to make possible
2:20:14
for re-indexation and to make possible
2:20:14
for re-indexation and to make possible eventual consistency and self-recovery
2:20:18
eventual consistency and self-recovery
2:20:18
eventual consistency and self-recovery the last one that i really want to
2:20:21
the last one that i really want to
2:20:21
the last one that i really want to briefly just mention is
2:20:22
briefly just mention is
2:20:22
briefly just mention is mirroring since our collections were not
2:20:25
mirroring since our collections were not
2:20:25
mirroring since our collections were not correctly partitioned from the very
2:20:27
correctly partitioned from the very
2:20:27
correctly partitioned from the very beginning we had the need of
2:20:29
beginning we had the need of
2:20:29
beginning we had the need of switching them at some point so in order
2:20:32
switching them at some point so in order
2:20:32
switching them at some point so in order to
2:20:33
to
2:20:33
to figure this out we added some more
2:20:35
figure this out we added some more
2:20:35
figure this out we added some more configuration
2:20:36
configuration
2:20:36
configuration that would tell that a collection has to
2:20:39
that would tell that a collection has to
2:20:39
that would tell that a collection has to be mirrored into a different collection
2:20:41
be mirrored into a different collection
2:20:41
be mirrored into a different collection and when any mutation happens creating a
2:20:44
and when any mutation happens creating a
2:20:44
and when any mutation happens creating a document modifying a document anything
2:20:47
document modifying a document anything
2:20:47
document modifying a document anything the outcome would be published to this
2:20:49
the outcome would be published to this
2:20:49
the outcome would be published to this other collection so as soon as we roll a
2:20:51
other collection so as soon as we roll a
2:20:51
other collection so as soon as we roll a new version with this configuration of
2:20:53
new version with this configuration of
2:20:53
new version with this configuration of our sdk over wrapper
2:20:55
our sdk over wrapper
2:20:55
our sdk over wrapper everything is going to be populating
2:20:56
everything is going to be populating
2:20:56
everything is going to be populating automatically so the only thing that is
2:20:58
automatically so the only thing that is
2:20:58
automatically so the only thing that is left
2:20:59
left
2:20:59
left is just to run a backfill
2:21:02
is just to run a backfill
2:21:02
is just to run a backfill function that would create all of their
2:21:05
function that would create all of their
2:21:05
function that would create all of their remaining
2:21:06
remaining
2:21:06
remaining items going through each of the items in
2:21:09
items going through each of the items in
2:21:09
items going through each of the items in the collection
2:21:10
the collection
2:21:10
the collection into the new collection with the right
2:21:12
into the new collection with the right
2:21:12
into the new collection with the right partition
2:21:13
partition
2:21:13
partition then we reach the point where we have
2:21:14
then we reach the point where we have
2:21:14
then we reach the point where we have one to one
2:21:17
one to one
2:21:17
one to one documents between both uh collections
2:21:19
documents between both uh collections
2:21:19
documents between both uh collections and the only thing remaining would be to
2:21:21
and the only thing remaining would be to
2:21:21
and the only thing remaining would be to switch reads
2:21:22
switch reads
2:21:22
switch reads to the new collection this was very very
2:21:24
to the new collection this was very very
2:21:24
to the new collection this was very very helpful and it helps to
2:21:26
helpful and it helps to
2:21:26
helpful and it helps to correct our data in production without
2:21:30
correct our data in production without
2:21:30
correct our data in production without downtime and remodel our collection of
2:21:32
downtime and remodel our collection of
2:21:32
downtime and remodel our collection of course there is much more to this
2:21:34
course there is much more to this
2:21:34
course there is much more to this it's a very condensated talk so feel
2:21:36
it's a very condensated talk so feel
2:21:36
it's a very condensated talk so feel free to reach out
2:21:37
free to reach out
2:21:37
free to reach out and discuss ideas and
2:21:40
and discuss ideas and
2:21:40
and discuss ideas and and any other questions you might have
2:21:43
and any other questions you might have
2:21:43
and any other questions you might have and if you want to help us build a
2:21:45
and if you want to help us build a
2:21:45
and if you want to help us build a better faster
2:21:46
better faster
2:21:46
better faster more scalable web we are hiring so reach
2:21:49
more scalable web we are hiring so reach
2:21:49
more scalable web we are hiring so reach out to barcelona.com
2:21:51
out to barcelona.com
2:21:51
out to barcelona.com jobs or ping that me directly in twitter
2:21:54
jobs or ping that me directly in twitter
2:21:54
jobs or ping that me directly in twitter at javi velasco
2:21:56
at javi velasco
2:21:56
at javi velasco my dm's are open thank you so much
2:22:05
thank you so much javi for a great talk
2:22:08
thank you so much javi for a great talk
2:22:08
thank you so much javi for a great talk i really liked how you uh really focused
2:22:10
i really liked how you uh really focused
2:22:10
i really liked how you uh really focused on how the wrappers help
2:22:12
on how the wrappers help
2:22:12
on how the wrappers help when there's changes upstream done to
2:22:14
when there's changes upstream done to
2:22:14
when there's changes upstream done to the sdk and you want to kind of protect
2:22:16
the sdk and you want to kind of protect
2:22:16
the sdk and you want to kind of protect your users and protect your apps and
2:22:18
your users and protect your apps and
2:22:18
your users and protect your apps and and decide how you want the change to go
2:22:20
and decide how you want the change to go
2:22:20
and decide how you want the change to go downstream to your applications gives
2:22:21
downstream to your applications gives
2:22:21
downstream to your applications gives you a lot of flexibility that way i
2:22:22
you a lot of flexibility that way i
2:22:22
you a lot of flexibility that way i thought that was really cool
2:22:24
thought that was really cool
2:22:24
thought that was really cool all right thank you one question we got
2:22:26
all right thank you one question we got
2:22:26
all right thank you one question we got from our users was
2:22:28
from our users was
2:22:28
from our users was um are there cases where you don't want
2:22:31
um are there cases where you don't want
2:22:31
um are there cases where you don't want to use wrappers where
2:22:33
to use wrappers where
2:22:33
to use wrappers where it's not worth doing so because of
2:22:34
it's not worth doing so because of
2:22:34
it's not worth doing so because of performance or
2:22:36
performance or
2:22:36
performance or or some other reason we're not thinking
2:22:37
or some other reason we're not thinking
2:22:37
or some other reason we're not thinking of right
2:22:39
of right
2:22:39
of right i think that in general in every
2:22:42
i think that in general in every
2:22:42
i think that in general in every api that is out of your control but not
2:22:45
api that is out of your control but not
2:22:45
api that is out of your control but not necessarily just tooling but
2:22:47
necessarily just tooling but
2:22:47
necessarily just tooling but more like sdks for github for example
2:22:50
more like sdks for github for example
2:22:50
more like sdks for github for example on other git providers we are also
2:22:53
on other git providers we are also
2:22:53
on other git providers we are also having like our own clients that
2:22:55
having like our own clients that
2:22:55
having like our own clients that sometimes are relying on their sdks and
2:22:57
sometimes are relying on their sdks and
2:22:57
sometimes are relying on their sdks and sometimes are
2:22:58
sometimes are
2:22:58
sometimes are just doing raw requests in those cases
2:23:01
just doing raw requests in those cases
2:23:01
just doing raw requests in those cases if some new change comes from new api
2:23:04
if some new change comes from new api
2:23:04
if some new change comes from new api experimental api comes in we don't even
2:23:07
experimental api comes in we don't even
2:23:07
experimental api comes in we don't even have to wait
2:23:08
have to wait
2:23:08
have to wait for them to roll those into the sdk we
2:23:11
for them to roll those into the sdk we
2:23:11
for them to roll those into the sdk we can just go and implement it
2:23:13
can just go and implement it
2:23:13
can just go and implement it or on uh that is very cool cases where
2:23:16
or on uh that is very cool cases where
2:23:16
or on uh that is very cool cases where we don't really need this i guess that
2:23:18
we don't really need this i guess that
2:23:18
we don't really need this i guess that if you're using a library that is very
2:23:19
if you're using a library that is very
2:23:19
if you're using a library that is very lightweight like for example if you are
2:23:21
lightweight like for example if you are
2:23:21
lightweight like for example if you are using it
2:23:22
using it
2:23:22
using it node.js would be something like low dash
2:23:24
node.js would be something like low dash
2:23:24
node.js would be something like low dash or ramda or something like that that is
2:23:26
or ramda or something like that that is
2:23:26
or ramda or something like that that is very very minimal
2:23:27
very very minimal
2:23:27
very very minimal those cases they are just helpers so
2:23:29
those cases they are just helpers so
2:23:29
those cases they are just helpers so there is no really actual value
2:23:30
there is no really actual value
2:23:30
there is no really actual value having a wrapper on those
2:23:34
having a wrapper on those
2:23:34
having a wrapper on those okay great so it sounds like assessing
2:23:35
okay great so it sounds like assessing
2:23:35
okay great so it sounds like assessing the value is is kind of like a balance
2:23:37
the value is is kind of like a balance
2:23:37
the value is is kind of like a balance between
2:23:38
between
2:23:38
between are we getting any benefits out of you
2:23:40
are we getting any benefits out of you
2:23:40
are we getting any benefits out of you know putting this layer between for our
2:23:42
know putting this layer between for our
2:23:42
know putting this layer between for our users
2:23:43
users
2:23:43
users yeah the primary one the most important
2:23:45
yeah the primary one the most important
2:23:45
yeah the primary one the most important one i would say
2:23:46
one i would say
2:23:46
one i would say is observability if you have a
2:23:49
is observability if you have a
2:23:49
is observability if you have a synchronous function in javascript for
2:23:50
synchronous function in javascript for
2:23:50
synchronous function in javascript for example there's no need to observe that
2:23:52
example there's no need to observe that
2:23:52
example there's no need to observe that usually depends on the size and what
2:23:54
usually depends on the size and what
2:23:54
usually depends on the size and what it's doing of course
2:23:55
it's doing of course
2:23:56
it's doing of course but if you are doing under the corpus on
2:23:57
but if you are doing under the corpus on
2:23:57
but if you are doing under the corpus on networks request like for example
2:23:59
networks request like for example
2:23:59
networks request like for example something that we
2:23:59
something that we
2:24:00
something that we have found very valuable is that the sdk
2:24:02
have found very valuable is that the sdk
2:24:02
have found very valuable is that the sdk is doing its own retry strategy under
2:24:04
is doing its own retry strategy under
2:24:04
is doing its own retry strategy under the covers
2:24:05
the covers
2:24:05
the covers so if we just wrap it and we just trace
2:24:08
so if we just wrap it and we just trace
2:24:08
so if we just wrap it and we just trace that we are going to see in the end how
2:24:10
that we are going to see in the end how
2:24:10
that we are going to see in the end how much
2:24:10
much
2:24:10
much the sdk call took but not how much
2:24:13
the sdk call took but not how much
2:24:13
the sdk call took but not how much the request took so that is why these
2:24:17
the request took so that is why these
2:24:17
the request took so that is why these plugins were very very useful and this
2:24:19
plugins were very very useful and this
2:24:20
plugins were very very useful and this actually extends
2:24:21
actually extends
2:24:21
actually extends to other sdks you can maybe disable
2:24:25
to other sdks you can maybe disable
2:24:25
to other sdks you can maybe disable um the retry strategy if you are not so
2:24:27
um the retry strategy if you are not so
2:24:28
um the retry strategy if you are not so sure about it
2:24:29
sure about it
2:24:29
sure about it and put your own in or your own in front
2:24:31
and put your own in or your own in front
2:24:31
and put your own in or your own in front of it where you can also instrument
2:24:33
of it where you can also instrument
2:24:33
of it where you can also instrument proper observability so i think that's
2:24:35
proper observability so i think that's
2:24:35
proper observability so i think that's very helpful
2:24:37
very helpful
2:24:37
very helpful yeah that's great that makes a lot of
2:24:38
yeah that's great that makes a lot of
2:24:38
yeah that's great that makes a lot of sense um
2:24:40
sense um
2:24:40
sense um um just sorry one more question um
2:24:43
um just sorry one more question um
2:24:43
um just sorry one more question um thanks so much javi i i really enjoyed
2:24:45
thanks so much javi i i really enjoyed
2:24:45
thanks so much javi i i really enjoyed this um presentation
2:24:47
this um presentation
2:24:47
this um presentation definitely a model a model partner here
2:24:50
definitely a model a model partner here
2:24:50
definitely a model a model partner here and how to implement
2:24:51
and how to implement
2:24:51
and how to implement use all the functions all the things we
2:24:52
use all the functions all the things we
2:24:52
use all the functions all the things we have available now quick question you
2:24:55
have available now quick question you
2:24:55
have available now quick question you mentioned that um for partitioning you
2:24:57
mentioned that um for partitioning you
2:24:57
mentioned that um for partitioning you you had the wrong partitioning strategy
2:24:58
you had the wrong partitioning strategy
2:24:58
you had the wrong partitioning strategy how important was observability in
2:25:00
how important was observability in
2:25:00
how important was observability in diagnosing hey
2:25:01
diagnosing hey
2:25:02
diagnosing hey and maybe i have the wrong partition
2:25:04
and maybe i have the wrong partition
2:25:04
and maybe i have the wrong partition here
2:25:05
here
2:25:05
here oh that was that is a very good question
2:25:07
oh that was that is a very good question
2:25:07
oh that was that is a very good question thank you
2:25:08
thank you
2:25:08
thank you so uh once we have the observabilities
2:25:11
so uh once we have the observabilities
2:25:11
so uh once we have the observabilities is when
2:25:11
is when
2:25:11
is when it actually started popping out the
2:25:13
it actually started popping out the
2:25:13
it actually started popping out the first thing that we have seen back in
2:25:15
first thing that we have seen back in
2:25:15
first thing that we have seen back in 2017
2:25:16
2017
2:25:16
2017 was that when we were running
2:25:18
was that when we were running
2:25:18
was that when we were running cross-partition query
2:25:19
cross-partition query
2:25:19
cross-partition query we added this wrapper that was just the
2:25:22
we added this wrapper that was just the
2:25:22
we added this wrapper that was just the logger that was a very naive
2:25:23
logger that was a very naive
2:25:24
logger that was a very naive implementation
2:25:25
implementation
2:25:25
implementation but then we visited the logs of a very
2:25:27
but then we visited the logs of a very
2:25:27
but then we visited the logs of a very critical service
2:25:28
critical service
2:25:28
critical service and for every request we have seen like
2:25:30
and for every request we have seen like
2:25:30
and for every request we have seen like thousands of log lines
2:25:32
thousands of log lines
2:25:32
thousands of log lines so that actually revealed that there was
2:25:34
so that actually revealed that there was
2:25:34
so that actually revealed that there was an issue there we are
2:25:36
an issue there we are
2:25:36
an issue there we are using a very an access pattern that is
2:25:38
using a very an access pattern that is
2:25:38
using a very an access pattern that is very required to be
2:25:40
very required to be
2:25:40
very required to be fast and optimal and it was really
2:25:43
fast and optimal and it was really
2:25:43
fast and optimal and it was really terrible
2:25:43
terrible
2:25:44
terrible so that is one of the ways that we
2:25:45
so that is one of the ways that we
2:25:45
so that is one of the ways that we discovered that then when we got
2:25:47
discovered that then when we got
2:25:48
discovered that then when we got something more sophisticated
2:25:49
something more sophisticated
2:25:49
something more sophisticated it was even more explicit because you
2:25:51
it was even more explicit because you
2:25:52
it was even more explicit because you can actually
2:25:52
can actually
2:25:52
can actually see the flame graph showing you a lot of
2:25:56
see the flame graph showing you a lot of
2:25:56
see the flame graph showing you a lot of tiny slots
2:25:57
tiny slots
2:25:57
tiny slots with every single request that it is
2:25:59
with every single request that it is
2:25:59
with every single request that it is doing under the covers so observability
2:26:01
doing under the covers so observability
2:26:01
doing under the covers so observability was
2:26:02
was
2:26:02
was super important for that and
2:26:05
super important for that and
2:26:05
super important for that and this solution that we came up with the
2:26:07
this solution that we came up with the
2:26:07
this solution that we came up with the secondary keys
2:26:08
secondary keys
2:26:08
secondary keys was very very helpful especially in our
2:26:11
was very very helpful especially in our
2:26:11
was very very helpful especially in our dashboard
2:26:12
dashboard
2:26:12
dashboard when fetching users we had very common
2:26:15
when fetching users we had very common
2:26:15
when fetching users we had very common use cases where we needed to fetch a
2:26:17
use cases where we needed to fetch a
2:26:17
use cases where we needed to fetch a user by email while the email is not the
2:26:19
user by email while the email is not the
2:26:19
user by email while the email is not the partition key
2:26:20
partition key
2:26:20
partition key so how we how can we do that ensuring
2:26:22
so how we how can we do that ensuring
2:26:22
so how we how can we do that ensuring uniqueness and ensuring that it works
2:26:24
uniqueness and ensuring that it works
2:26:24
uniqueness and ensuring that it works in a very fast and efficient way it was
2:26:26
in a very fast and efficient way it was
2:26:26
in a very fast and efficient way it was very difficult and
2:26:27
very difficult and
2:26:28
very difficult and in a previous talk it was mentioned that
2:26:30
in a previous talk it was mentioned that
2:26:30
in a previous talk it was mentioned that since you're lacking transaction heal
2:26:32
since you're lacking transaction heal
2:26:32
since you're lacking transaction heal because you are using a non-sql database
2:26:35
because you are using a non-sql database
2:26:35
because you are using a non-sql database you have to implement our
2:26:36
you have to implement our
2:26:36
you have to implement our own kind of transactions and that was a
2:26:39
own kind of transactions and that was a
2:26:39
own kind of transactions and that was a very challenging part
2:26:40
very challenging part
2:26:40
very challenging part but it's definitely worth it
2:26:45
okay awesome i i actually had a bunch of
2:26:47
okay awesome i i actually had a bunch of
2:26:48
okay awesome i i actually had a bunch of questions but we're not going to have
2:26:48
questions but we're not going to have
2:26:48
questions but we're not going to have time
2:26:50
time
2:26:50
time so maybe i'll reach out to you have a
2:26:52
so maybe i'll reach out to you have a
2:26:52
so maybe i'll reach out to you have a and i'm sure others others will as well
2:26:54
and i'm sure others others will as well
2:26:54
and i'm sure others others will as well there's
2:26:55
there's
2:26:55
there's some really cool features that you built
2:26:56
some really cool features that you built
2:26:56
some really cool features that you built there so that's awesome
2:26:58
there so that's awesome
2:26:58
there so that's awesome um thank you so thank you thank you for
2:27:01
um thank you so thank you thank you for
2:27:01
um thank you so thank you thank you for that um
2:27:03
that um
2:27:03
that um and i've i believe we have now reached
2:27:05
and i've i believe we have now reached
2:27:05
and i've i believe we have now reached the
2:27:06
the
2:27:06
the last session of the azure cosmos db conf
2:27:09
last session of the azure cosmos db conf
2:27:09
last session of the azure cosmos db conf 2021 so thanks to all the speakers that
2:27:11
2021 so thanks to all the speakers that
2:27:11
2021 so thanks to all the speakers that have gone
2:27:12
have gone
2:27:12
have gone before we have one session left and i'm
2:27:15
before we have one session left and i'm
2:27:15
before we have one session left and i'm delighted
2:27:16
delighted
2:27:16
delighted to welcome um gary and amrish from
2:27:19
to welcome um gary and amrish from
2:27:19
to welcome um gary and amrish from our friends asos who are going to talk
2:27:20
our friends asos who are going to talk
2:27:20
our friends asos who are going to talk about operational triumph
2:27:23
about operational triumph
2:27:23
about operational triumph with azure cosmos db take it away amrish
2:27:26
with azure cosmos db take it away amrish
2:27:26
with azure cosmos db take it away amrish uh thanks yeah so hello everyone um some
2:27:29
uh thanks yeah so hello everyone um some
2:27:29
uh thanks yeah so hello everyone um some really great sessions today i definitely
2:27:31
really great sessions today i definitely
2:27:31
really great sessions today i definitely learnt
2:27:31
learnt
2:27:31
learnt uh loads um so hi i'm amrish uh welcome
2:27:35
uh loads um so hi i'm amrish uh welcome
2:27:35
uh loads um so hi i'm amrish uh welcome to our session on operational triumph
2:27:37
to our session on operational triumph
2:27:37
to our session on operational triumph with cosmos db
2:27:38
with cosmos db
2:27:38
with cosmos db uh where we'll look at how asos has been
2:27:41
uh where we'll look at how asos has been
2:27:41
uh where we'll look at how asos has been using cosmos db to achieve success i am
2:27:44
using cosmos db to achieve success i am
2:27:44
using cosmos db to achieve success i am a solution architect here at asos
2:27:46
a solution architect here at asos
2:27:46
a solution architect here at asos looking for
2:27:47
looking for
2:27:47
looking for looking after the order and return
2:27:49
looking after the order and return
2:27:49
looking after the order and return domains i've got here with me my
2:27:51
domains i've got here with me my
2:27:51
domains i've got here with me my uh my colleague gary who will introduce
2:27:54
uh my colleague gary who will introduce
2:27:54
uh my colleague gary who will introduce himself
2:27:55
himself
2:27:55
himself hi i'm gary strange big data architect
2:27:58
hi i'm gary strange big data architect
2:27:58
hi i'm gary strange big data architect at asos
2:27:59
at asos
2:27:59
at asos i work alongside amrish and other
2:28:01
i work alongside amrish and other
2:28:01
i work alongside amrish and other architects delivering
2:28:03
architects delivering
2:28:03
architects delivering data data solutions
2:28:06
so a little bit about asus asos is a
2:28:09
so a little bit about asus asos is a
2:28:09
so a little bit about asus asos is a globally distributed online fashion
2:28:11
globally distributed online fashion
2:28:11
globally distributed online fashion retailer whose aim is to be the number
2:28:13
retailer whose aim is to be the number
2:28:13
retailer whose aim is to be the number one destination for fashion loving 20
2:28:15
one destination for fashion loving 20
2:28:15
one destination for fashion loving 20 somethings
2:28:15
somethings
2:28:16
somethings providing a market-leading app and web
2:28:18
providing a market-leading app and web
2:28:18
providing a market-leading app and web experience
2:28:19
experience
2:28:19
experience since its official release in 2017 asos
2:28:22
since its official release in 2017 asos
2:28:22
since its official release in 2017 asos has been
2:28:23
has been
2:28:23
has been using cosmos db and all of its different
2:28:26
using cosmos db and all of its different
2:28:26
using cosmos db and all of its different features
2:28:26
features
2:28:26
features to re-imagine our commerce platform from
2:28:29
to re-imagine our commerce platform from
2:28:29
to re-imagine our commerce platform from millisecond slas
2:28:30
millisecond slas
2:28:30
millisecond slas to the turnkey distribution and infinite
2:28:33
to the turnkey distribution and infinite
2:28:33
to the turnkey distribution and infinite scaling potential
2:28:35
scaling potential
2:28:35
scaling potential so let's look at how cosmos db has been
2:28:37
so let's look at how cosmos db has been
2:28:37
so let's look at how cosmos db has been vital
2:28:38
vital
2:28:38
vital to our you know to be able to conquer
2:28:40
to our you know to be able to conquer
2:28:40
to our you know to be able to conquer our operational data needs
2:28:42
our operational data needs
2:28:42
our operational data needs um while remaining flexible for peak
2:28:44
um while remaining flexible for peak
2:28:44
um while remaining flexible for peak periods uh such as
2:28:46
periods uh such as
2:28:46
periods uh such as black friday um as everyone knows
2:28:49
black friday um as everyone knows
2:28:49
black friday um as everyone knows we operate at huge scale and a global
2:28:52
we operate at huge scale and a global
2:28:52
we operate at huge scale and a global scale
2:28:53
scale
2:28:53
scale so let's let's take a look at some
2:28:54
so let's let's take a look at some
2:28:54
so let's let's take a look at some numbers and just to understand
2:28:56
numbers and just to understand
2:28:56
numbers and just to understand what that really means and how we use
2:28:58
what that really means and how we use
2:28:58
what that really means and how we use cosmos and the rest of the azure
2:29:00
cosmos and the rest of the azure
2:29:00
cosmos and the rest of the azure ecosystem
2:29:01
ecosystem
2:29:01
ecosystem to help us achieve that global scale so
2:29:04
to help us achieve that global scale so
2:29:04
to help us achieve that global scale so we offer our app and web experience
2:29:08
we offer our app and web experience
2:29:08
we offer our app and web experience in 10 languages across 200 markets
2:29:11
in 10 languages across 200 markets
2:29:11
in 10 languages across 200 markets we have over 85 000 products on the site
2:29:14
we have over 85 000 products on the site
2:29:14
we have over 85 000 products on the site at any time
2:29:15
at any time
2:29:15
at any time um 5 000 new products added every week
2:29:18
um 5 000 new products added every week
2:29:18
um 5 000 new products added every week and we have an active
2:29:19
and we have an active
2:29:19
and we have an active uh customer base of 24.9 million
2:29:23
uh customer base of 24.9 million
2:29:23
uh customer base of 24.9 million customers
2:29:24
customers
2:29:24
customers um in the financial year 20 just gone um
2:29:27
um in the financial year 20 just gone um
2:29:27
um in the financial year 20 just gone um we actually processed 80.2 million
2:29:30
we actually processed 80.2 million
2:29:30
we actually processed 80.2 million orders
2:29:32
orders
2:29:32
orders let's have a look at um how we're going
2:29:34
let's have a look at um how we're going
2:29:34
let's have a look at um how we're going where and in our ecosystem
2:29:37
where and in our ecosystem
2:29:37
where and in our ecosystem we use cosmos db
2:29:40
we use cosmos db
2:29:40
we use cosmos db so we use a micro service architecture
2:29:43
so we use a micro service architecture
2:29:43
so we use a micro service architecture which is made up of many many services
2:29:45
which is made up of many many services
2:29:45
which is made up of many many services some of which you can see
2:29:46
some of which you can see
2:29:46
some of which you can see on the screen today so our order service
2:29:48
on the screen today so our order service
2:29:48
on the screen today so our order service being the one that
2:29:50
being the one that
2:29:50
being the one that where customers can place orders our
2:29:52
where customers can place orders our
2:29:52
where customers can place orders our pricing
2:29:53
pricing
2:29:53
pricing system where we can manage our prices
2:29:56
system where we can manage our prices
2:29:56
system where we can manage our prices for all of our different
2:29:57
for all of our different
2:29:57
for all of our different uh products payment service which we
2:30:00
uh products payment service which we
2:30:00
uh products payment service which we use to take payments and transact with
2:30:03
use to take payments and transact with
2:30:03
use to take payments and transact with all of our different payment providers
2:30:05
all of our different payment providers
2:30:05
all of our different payment providers stock and fulfillment services
2:30:07
stock and fulfillment services
2:30:07
stock and fulfillment services where we can manage stock availability
2:30:09
where we can manage stock availability
2:30:09
where we can manage stock availability as well as manage
2:30:10
as well as manage
2:30:10
as well as manage integrations with carrier management
2:30:12
integrations with carrier management
2:30:12
integrations with carrier management systems and various other aspects of
2:30:14
systems and various other aspects of
2:30:14
systems and various other aspects of fulfillment
2:30:15
fulfillment
2:30:15
fulfillment um our promotional system is you know
2:30:18
um our promotional system is you know
2:30:18
um our promotional system is you know heavily utilized
2:30:19
heavily utilized
2:30:19
heavily utilized especially for usage and
2:30:23
especially for usage and
2:30:23
especially for usage and we use cosmos heavily in this area our
2:30:25
we use cosmos heavily in this area our
2:30:25
we use cosmos heavily in this area our saved item service is one of the largest
2:30:28
saved item service is one of the largest
2:30:28
saved item service is one of the largest consumers of cosmos db
2:30:30
consumers of cosmos db
2:30:30
consumers of cosmos db and provides a fantastic experience for
2:30:32
and provides a fantastic experience for
2:30:32
and provides a fantastic experience for customers where they can save
2:30:34
customers where they can save
2:30:34
customers where they can save all of the different items that um they
2:30:36
all of the different items that um they
2:30:36
all of the different items that um they want to
2:30:38
want to
2:30:38
want to purchase or they can create boards and
2:30:40
purchase or they can create boards and
2:30:40
purchase or they can create boards and share them with their
2:30:41
share them with their
2:30:41
share them with their friends and family um we're gonna look
2:30:44
friends and family um we're gonna look
2:30:44
friends and family um we're gonna look at why we use cosmos
2:30:45
at why we use cosmos
2:30:46
at why we use cosmos db and just to better understand all of
2:30:49
db and just to better understand all of
2:30:49
db and just to better understand all of the different aspects around
2:30:50
the different aspects around
2:30:50
the different aspects around why we prefer this so first and foremost
2:30:53
why we prefer this so first and foremost
2:30:53
why we prefer this so first and foremost we love
2:30:54
we love
2:30:54
we love paz and cosmos db being a platform as a
2:30:56
paz and cosmos db being a platform as a
2:30:56
paz and cosmos db being a platform as a service offering
2:30:57
service offering
2:30:57
service offering helps us reduce our operational
2:30:59
helps us reduce our operational
2:30:59
helps us reduce our operational overheads as well as
2:31:00
overheads as well as
2:31:00
overheads as well as um being cost optimal uh when we want to
2:31:04
um being cost optimal uh when we want to
2:31:04
um being cost optimal uh when we want to run
2:31:04
run
2:31:04
run our production or most mission critical
2:31:07
our production or most mission critical
2:31:07
our production or most mission critical uh workloads
2:31:08
uh workloads
2:31:08
uh workloads um we love nines here at asos so the
2:31:11
um we love nines here at asos so the
2:31:11
um we love nines here at asos so the fact that a cosmos db
2:31:13
fact that a cosmos db
2:31:13
fact that a cosmos db can provide up to five nines of
2:31:16
can provide up to five nines of
2:31:16
can provide up to five nines of availability
2:31:17
availability
2:31:17
availability is a huge deal when it comes to
2:31:19
is a huge deal when it comes to
2:31:19
is a huge deal when it comes to providing that great customer experience
2:31:21
providing that great customer experience
2:31:21
providing that great customer experience that
2:31:21
that
2:31:21
that everybody knows us for
2:31:25
everybody knows us for
2:31:25
everybody knows us for being schemerless provides us with
2:31:27
being schemerless provides us with
2:31:27
being schemerless provides us with business agility being able to
2:31:28
business agility being able to
2:31:28
business agility being able to adapt and change to business uh
2:31:31
adapt and change to business uh
2:31:31
adapt and change to business uh requirements and environments
2:31:33
requirements and environments
2:31:33
requirements and environments um it's scalable and highly performant
2:31:35
um it's scalable and highly performant
2:31:35
um it's scalable and highly performant and the
2:31:36
and the
2:31:36
and the distribution of data the data
2:31:38
distribution of data the data
2:31:38
distribution of data the data partitioning is made
2:31:39
partitioning is made
2:31:40
partitioning is made easy and because you know provides it
2:31:42
easy and because you know provides it
2:31:42
easy and because you know provides it out of the box
2:31:43
out of the box
2:31:43
out of the box and we don't have to to think but to
2:31:45
and we don't have to to think but to
2:31:45
and we don't have to to think but to think too carefully
2:31:46
think too carefully
2:31:46
think too carefully about this but um having a great
2:31:49
about this but um having a great
2:31:49
about this but um having a great partitioning strategy
2:31:51
partitioning strategy
2:31:51
partitioning strategy is super important and the latency
2:31:53
is super important and the latency
2:31:53
is super important and the latency guarantees on read and write operations
2:31:55
guarantees on read and write operations
2:31:55
guarantees on read and write operations helps us
2:31:56
helps us
2:31:56
helps us focus and fine tune all of that
2:31:57
focus and fine tune all of that
2:31:57
focus and fine tune all of that performance that we need down to that 10
2:31:59
performance that we need down to that 10
2:31:59
performance that we need down to that 10 millisecond
2:32:02
millisecond
2:32:02
millisecond sla and the change feed has been a real
2:32:04
sla and the change feed has been a real
2:32:04
sla and the change feed has been a real game changer for us
2:32:05
game changer for us
2:32:05
game changer for us to adapt and re-imagine our
2:32:09
to adapt and re-imagine our
2:32:09
to adapt and re-imagine our architecture especially in the commerce
2:32:11
architecture especially in the commerce
2:32:11
architecture especially in the commerce area
2:32:12
area
2:32:12
area next we'll look at a use case around
2:32:15
next we'll look at a use case around
2:32:15
next we'll look at a use case around owner management
2:32:16
owner management
2:32:16
owner management and how we can utilize cosmos db to re
2:32:19
and how we can utilize cosmos db to re
2:32:19
and how we can utilize cosmos db to re re-architect the system to be
2:32:23
re-architect the system to be
2:32:23
re-architect the system to be operational globally as well as being
2:32:25
operational globally as well as being
2:32:25
operational globally as well as being resilient and scalable and highly
2:32:27
resilient and scalable and highly
2:32:27
resilient and scalable and highly performant
2:32:29
performant
2:32:29
performant so now that we know where and why
2:32:32
so now that we know where and why
2:32:32
so now that we know where and why we use cosmos db let's take a look at
2:32:34
we use cosmos db let's take a look at
2:32:34
we use cosmos db let's take a look at the order service in more detail
2:32:36
the order service in more detail
2:32:36
the order service in more detail and how we can leverage uh cosmos tv to
2:32:39
and how we can leverage uh cosmos tv to
2:32:39
and how we can leverage uh cosmos tv to build a cloud native event driven order
2:32:40
build a cloud native event driven order
2:32:40
build a cloud native event driven order processing
2:32:41
processing
2:32:42
processing engine in azure so the first thing that
2:32:44
engine in azure so the first thing that
2:32:44
engine in azure so the first thing that we did is we broke that capability into
2:32:46
we did is we broke that capability into
2:32:46
we did is we broke that capability into two logical parts
2:32:48
two logical parts
2:32:48
two logical parts the first being our order booking
2:32:49
the first being our order booking
2:32:49
the first being our order booking capability and here it's super important
2:32:52
capability and here it's super important
2:32:52
capability and here it's super important that
2:32:53
that
2:32:53
that customers can continue to take
2:32:56
customers can continue to take
2:32:56
customers can continue to take continue to keep placing orders and is
2:32:59
continue to keep placing orders and is
2:32:59
continue to keep placing orders and is available globally
2:33:01
available globally
2:33:01
available globally so that even in the event of a regional
2:33:03
so that even in the event of a regional
2:33:03
so that even in the event of a regional outage we can continue
2:33:05
outage we can continue
2:33:05
outage we can continue our operations so cosmos db provides all
2:33:08
our operations so cosmos db provides all
2:33:08
our operations so cosmos db provides all of this capability to us
2:33:10
of this capability to us
2:33:10
of this capability to us um with that industry-leading dual
2:33:12
um with that industry-leading dual
2:33:12
um with that industry-leading dual redundancy and
2:33:13
redundancy and
2:33:13
redundancy and multi-master right features next
2:33:16
multi-master right features next
2:33:16
multi-master right features next once we've got all of that data we can
2:33:18
once we've got all of that data we can
2:33:18
once we've got all of that data we can persist all of those requests into
2:33:21
persist all of those requests into
2:33:21
persist all of those requests into an azure cosmos db and then once that is
2:33:23
an azure cosmos db and then once that is
2:33:23
an azure cosmos db and then once that is there we've got the change feed
2:33:25
there we've got the change feed
2:33:25
there we've got the change feed capability
2:33:26
capability
2:33:26
capability which can then hand off to our order
2:33:28
which can then hand off to our order
2:33:28
which can then hand off to our order processing capability
2:33:30
processing capability
2:33:30
processing capability so let's have a look at order processing
2:33:32
so let's have a look at order processing
2:33:32
so let's have a look at order processing so order processing we've
2:33:33
so order processing we've
2:33:34
so order processing we've re-architected or are continuously
2:33:36
re-architected or are continuously
2:33:36
re-architected or are continuously re-architecting
2:33:37
re-architecting
2:33:37
re-architecting into a serverless event-driven
2:33:40
into a serverless event-driven
2:33:40
into a serverless event-driven architecture
2:33:41
architecture
2:33:41
architecture here you can see the change feed
2:33:43
here you can see the change feed
2:33:43
here you can see the change feed processing those
2:33:44
processing those
2:33:44
processing those those orders coming in and emitting out
2:33:47
those orders coming in and emitting out
2:33:47
those orders coming in and emitting out an event
2:33:48
an event
2:33:48
an event and once that event is uh published it
2:33:50
and once that event is uh published it
2:33:50
and once that event is uh published it goes on to
2:33:52
goes on to
2:33:52
goes on to our azure service bus name spaces where
2:33:55
our azure service bus name spaces where
2:33:55
our azure service bus name spaces where a whole orchestration and workflow is
2:33:57
a whole orchestration and workflow is
2:33:57
a whole orchestration and workflow is managed between all the different
2:34:00
managed between all the different
2:34:00
managed between all the different micro services that we have whether
2:34:01
micro services that we have whether
2:34:02
micro services that we have whether that's stock allocation
2:34:03
that's stock allocation
2:34:03
that's stock allocation whether that's payment integrations
2:34:05
whether that's payment integrations
2:34:05
whether that's payment integrations whether that's fulfillment decisions
2:34:07
whether that's fulfillment decisions
2:34:07
whether that's fulfillment decisions discount approvals fraud and the list
2:34:09
discount approvals fraud and the list
2:34:09
discount approvals fraud and the list goes on
2:34:10
goes on
2:34:10
goes on and all of that is highly scalable we
2:34:13
and all of that is highly scalable we
2:34:13
and all of that is highly scalable we can adapt and change that workflow
2:34:14
can adapt and change that workflow
2:34:14
can adapt and change that workflow however we want
2:34:16
however we want
2:34:16
however we want and all of those events being published
2:34:18
and all of those events being published
2:34:18
and all of those events being published generates a huge amount of data
2:34:20
generates a huge amount of data
2:34:20
generates a huge amount of data and all of that data can be event
2:34:21
and all of that data can be event
2:34:22
and all of that data can be event sourced into a cosmos db
2:34:24
sourced into a cosmos db
2:34:24
sourced into a cosmos db where we can then project out
2:34:28
where we can then project out
2:34:28
where we can then project out our read models and provide insights to
2:34:30
our read models and provide insights to
2:34:30
our read models and provide insights to the business to make
2:34:32
the business to make
2:34:32
the business to make decisions on how we want to improve
2:34:34
decisions on how we want to improve
2:34:34
decisions on how we want to improve performance or change the way that we
2:34:36
performance or change the way that we
2:34:36
performance or change the way that we operate
2:34:37
operate
2:34:37
operate um so all of that you know is
2:34:40
um so all of that you know is
2:34:40
um so all of that you know is is really really great having lots and
2:34:42
is really really great having lots and
2:34:42
is really really great having lots and lots of data and you can see here that
2:34:45
lots of data and you can see here that
2:34:45
lots of data and you can see here that cosmos is fundamental for us to be able
2:34:47
cosmos is fundamental for us to be able
2:34:47
cosmos is fundamental for us to be able to achieve all of that
2:34:48
to achieve all of that
2:34:48
to achieve all of that um now that we have the data um
2:34:51
um now that we have the data um
2:34:52
um now that we have the data um we need to provide some insights and so
2:34:54
we need to provide some insights and so
2:34:54
we need to provide some insights and so i'm gonna
2:34:55
i'm gonna
2:34:55
i'm gonna hand over to gary who will just talk us
2:34:57
hand over to gary who will just talk us
2:34:57
hand over to gary who will just talk us through about uh
2:34:58
through about uh
2:34:58
through about uh talk us through the asos data journey
2:35:01
talk us through the asos data journey
2:35:01
talk us through the asos data journey over to you gary thank you amrish um
2:35:04
over to you gary thank you amrish um
2:35:04
over to you gary thank you amrish um so we've had a lot of success with
2:35:07
so we've had a lot of success with
2:35:07
so we've had a lot of success with cosmos db
2:35:08
cosmos db
2:35:08
cosmos db in our operational microservice domains
2:35:11
in our operational microservice domains
2:35:11
in our operational microservice domains however as technologists you'll be
2:35:13
however as technologists you'll be
2:35:13
however as technologists you'll be acutely aware that we need
2:35:14
acutely aware that we need
2:35:14
acutely aware that we need data and data analytics to drive future
2:35:17
data and data analytics to drive future
2:35:17
data and data analytics to drive future business success
2:35:19
business success
2:35:19
business success so in the second half of this session um
2:35:22
so in the second half of this session um
2:35:22
so in the second half of this session um i'm going to present and share with you
2:35:23
i'm going to present and share with you
2:35:24
i'm going to present and share with you some of asos's data journey
2:35:26
some of asos's data journey
2:35:26
some of asos's data journey and what we've learned along the way and
2:35:28
and what we've learned along the way and
2:35:28
and what we've learned along the way and why we think
2:35:30
why we think
2:35:30
why we think cosmos and event hubs is actually a
2:35:32
cosmos and event hubs is actually a
2:35:32
cosmos and event hubs is actually a really great
2:35:33
really great
2:35:33
really great combo so so any
2:35:36
combo so so any
2:35:36
combo so so any data journey has to start with some data
2:35:40
data journey has to start with some data
2:35:40
data journey has to start with some data pipeline and ingestion so i'll start
2:35:42
pipeline and ingestion so i'll start
2:35:42
pipeline and ingestion so i'll start there picture ingestion
2:35:44
there picture ingestion
2:35:44
there picture ingestion so we've got this diverse family of
2:35:46
so we've got this diverse family of
2:35:46
so we've got this diverse family of microservices backed by cosmos db
2:35:49
microservices backed by cosmos db
2:35:49
microservices backed by cosmos db um and they're transacting high value
2:35:52
um and they're transacting high value
2:35:52
um and they're transacting high value enterprise messages between each other
2:35:55
enterprise messages between each other
2:35:55
enterprise messages between each other and we need to tap into that rich source
2:35:57
and we need to tap into that rich source
2:35:57
and we need to tap into that rich source of data and bring it into our
2:35:59
of data and bring it into our
2:35:59
of data and bring it into our enterprise analytics function
2:36:03
enterprise analytics function
2:36:03
enterprise analytics function and so we started out that journey a few
2:36:05
and so we started out that journey a few
2:36:05
and so we started out that journey a few years back and we've learned a lot along
2:36:07
years back and we've learned a lot along
2:36:07
years back and we've learned a lot along the way
2:36:09
the way
2:36:09
the way and before i get into the journey i just
2:36:10
and before i get into the journey i just
2:36:10
and before i get into the journey i just want to kind of dig in
2:36:12
want to kind of dig in
2:36:12
want to kind of dig in to a little more depth on what i mean by
2:36:15
to a little more depth on what i mean by
2:36:15
to a little more depth on what i mean by an enterprise analytics function
2:36:20
so we need to
2:36:22
so we need to
2:36:22
so we need to drive analytics for it through data um
2:36:26
drive analytics for it through data um
2:36:26
drive analytics for it through data um and then we didn't need to deliver on
2:36:27
and then we didn't need to deliver on
2:36:27
and then we didn't need to deliver on another number of facets
2:36:29
another number of facets
2:36:29
another number of facets so what is um high stake analytics well
2:36:32
so what is um high stake analytics well
2:36:32
so what is um high stake analytics well we use
2:36:33
we use
2:36:33
we use large volumes of data to train machine
2:36:36
large volumes of data to train machine
2:36:36
large volumes of data to train machine learning models and develop
2:36:38
learning models and develop
2:36:38
learning models and develop pricing strategies this gives the best
2:36:40
pricing strategies this gives the best
2:36:40
pricing strategies this gives the best price points customer
2:36:42
price points customer
2:36:42
price points customer and we're an agile business we're very
2:36:45
and we're an agile business we're very
2:36:45
and we're an agile business we're very agile with our approach to technology
2:36:47
agile with our approach to technology
2:36:47
agile with our approach to technology so we want to be able to iterate quickly
2:36:49
so we want to be able to iterate quickly
2:36:49
so we want to be able to iterate quickly and deliver
2:36:50
and deliver
2:36:50
and deliver small changes and observe feedback on
2:36:53
small changes and observe feedback on
2:36:53
small changes and observe feedback on the result
2:36:54
the result
2:36:54
the result and get some early results for example
2:36:57
and get some early results for example
2:36:57
and get some early results for example a b testing we want to be able to make
2:36:59
a b testing we want to be able to make
2:36:59
a b testing we want to be able to make sure that the businesses
2:37:01
sure that the businesses
2:37:01
sure that the businesses is healthy so we'll know that our
2:37:03
is healthy so we'll know that our
2:37:03
is healthy so we'll know that our customers are satisfied and they're
2:37:04
customers are satisfied and they're
2:37:04
customers are satisfied and they're getting experience they expect from asos
2:37:07
getting experience they expect from asos
2:37:07
getting experience they expect from asos uh and when we're running promotions we
2:37:10
uh and when we're running promotions we
2:37:10
uh and when we're running promotions we need to understand in real time that the
2:37:12
need to understand in real time that the
2:37:12
need to understand in real time that the promotion is working effectively
2:37:14
promotion is working effectively
2:37:14
promotion is working effectively and we can make tweaks as as the
2:37:17
and we can make tweaks as as the
2:37:17
and we can make tweaks as as the promotion is alive
2:37:19
promotion is alive
2:37:19
promotion is alive and we want to optimize our business we
2:37:21
and we want to optimize our business we
2:37:21
and we want to optimize our business we want to try and uh
2:37:22
want to try and uh
2:37:22
want to try and uh manage our return rates make sure we're
2:37:24
manage our return rates make sure we're
2:37:24
manage our return rates make sure we're not um
2:37:25
not um
2:37:25
not um not having too many returns on products
2:37:28
not having too many returns on products
2:37:28
not having too many returns on products um
2:37:28
um
2:37:28
um we want to make sure that our marketing
2:37:30
we want to make sure that our marketing
2:37:30
we want to make sure that our marketing spend is really effective
2:37:33
spend is really effective
2:37:33
spend is really effective and finally we want to grow our business
2:37:35
and finally we want to grow our business
2:37:35
and finally we want to grow our business we want to continue to grow our customer
2:37:37
we want to continue to grow our customer
2:37:37
we want to continue to grow our customer base
2:37:38
base
2:37:38
base and then and change and convert product
2:37:41
and then and change and convert product
2:37:41
and then and change and convert product searches
2:37:42
searches
2:37:42
searches into purchases so at the beginning of
2:37:45
into purchases so at the beginning of
2:37:45
into purchases so at the beginning of our our
2:37:46
our our
2:37:46
our our big data journey we um we relied on the
2:37:50
big data journey we um we relied on the
2:37:50
big data journey we um we relied on the convenience of that existing
2:37:52
convenience of that existing
2:37:52
convenience of that existing microsurface microservice architecture
2:37:55
microsurface microservice architecture
2:37:55
microsurface microservice architecture that amash has explained for us
2:37:57
that amash has explained for us
2:37:57
that amash has explained for us and it it was simple enough to sort of
2:38:00
and it it was simple enough to sort of
2:38:00
and it it was simple enough to sort of plug in
2:38:01
plug in
2:38:01
plug in um newly created ingestion services into
2:38:03
um newly created ingestion services into
2:38:03
um newly created ingestion services into that existing microservices
2:38:05
that existing microservices
2:38:05
that existing microservices infrastructure and i'll just explain
2:38:08
infrastructure and i'll just explain
2:38:08
infrastructure and i'll just explain quickly
2:38:09
quickly
2:38:09
quickly about how fundamentally these
2:38:12
about how fundamentally these
2:38:12
about how fundamentally these microservices generally operate in two
2:38:14
microservices generally operate in two
2:38:14
microservices generally operate in two modes so on the diagram here on the left
2:38:16
modes so on the diagram here on the left
2:38:16
modes so on the diagram here on the left we've got um a fat message based
2:38:19
we've got um a fat message based
2:38:19
we've got um a fat message based uh architecture so um
2:38:22
uh architecture so um
2:38:22
uh architecture so um document entity changes in microservices
2:38:25
document entity changes in microservices
2:38:25
document entity changes in microservices microservice a
2:38:26
microservice a
2:38:26
microservice a and microservice b needs to know about
2:38:28
and microservice b needs to know about
2:38:28
and microservice b needs to know about that the full entity is pushed onto a
2:38:30
that the full entity is pushed onto a
2:38:30
that the full entity is pushed onto a queue and then
2:38:31
queue and then
2:38:31
queue and then we have a subscriber to that queue that
2:38:33
we have a subscriber to that queue that
2:38:33
we have a subscriber to that queue that needs to take another action that the
2:38:35
needs to take another action that the
2:38:35
needs to take another action that the second microsoft is
2:38:37
second microsoft is
2:38:37
second microsoft is and then in the diagram on the right
2:38:39
and then in the diagram on the right
2:38:39
and then in the diagram on the right we've got um
2:38:40
we've got um
2:38:40
we've got um fin event so um customers change their
2:38:44
fin event so um customers change their
2:38:44
fin event so um customers change their profile or add something to bag
2:38:46
profile or add something to bag
2:38:46
profile or add something to bag we just send that event and say oh they
2:38:48
we just send that event and say oh they
2:38:48
we just send that event and say oh they changed their preference they checked
2:38:49
changed their preference they checked
2:38:49
changed their preference they checked they put
2:38:50
they put
2:38:50
they put product x in their bag and then the if
2:38:52
product x in their bag and then the if
2:38:52
product x in their bag and then the if the uh
2:38:54
the uh
2:38:54
the uh subsequent microservice needs to know
2:38:56
subsequent microservice needs to know
2:38:56
subsequent microservice needs to know about the complete state of the bag or
2:38:58
about the complete state of the bag or
2:38:58
about the complete state of the bag or the complete state of the car
2:38:59
the complete state of the car
2:38:59
the complete state of the car uh customer profile then make an api
2:39:01
uh customer profile then make an api
2:39:01
uh customer profile then make an api call back
2:39:02
call back
2:39:02
call back so i said we we plugged into these two
2:39:05
so i said we we plugged into these two
2:39:05
so i said we we plugged into these two kind these two
2:39:06
kind these two
2:39:06
kind these two flavors of of microservice architecture
2:39:09
flavors of of microservice architecture
2:39:09
flavors of of microservice architecture and that got us going
2:39:11
and that got us going
2:39:11
and that got us going uh in the early days but however we
2:39:14
uh in the early days but however we
2:39:14
uh in the early days but however we quickly discovered there were some cons
2:39:16
quickly discovered there were some cons
2:39:16
quickly discovered there were some cons to this approach
2:39:17
to this approach
2:39:17
to this approach and so the one big pro was utilizing
2:39:20
and so the one big pro was utilizing
2:39:20
and so the one big pro was utilizing existing
2:39:21
existing
2:39:21
existing uh assets as your assets and
2:39:22
uh assets as your assets and
2:39:22
uh assets as your assets and infrastructure but i also came with a
2:39:24
infrastructure but i also came with a
2:39:24
infrastructure but i also came with a com because we didn't really
2:39:26
com because we didn't really
2:39:26
com because we didn't really properly segregate our operational
2:39:28
properly segregate our operational
2:39:28
properly segregate our operational infrastructure from our
2:39:29
infrastructure from our
2:39:29
infrastructure from our um analytical workloads we want to have
2:39:33
um analytical workloads we want to have
2:39:33
um analytical workloads we want to have that segregation we want to be able to
2:39:35
that segregation we want to be able to
2:39:35
that segregation we want to be able to have effective management over our
2:39:37
have effective management over our
2:39:37
have effective management over our mission critical
2:39:38
mission critical
2:39:38
mission critical micro services and there's a lack of
2:39:42
micro services and there's a lack of
2:39:42
micro services and there's a lack of consistency
2:39:43
consistency
2:39:43
consistency uh across apis so we we're
2:39:46
uh across apis so we we're
2:39:46
uh across apis so we we're very as i said we're very agile with our
2:39:48
very as i said we're very agile with our
2:39:48
very as i said we're very agile with our tech we we have a lot of autonomy within
2:39:51
tech we we have a lot of autonomy within
2:39:51
tech we we have a lot of autonomy within the teams
2:39:51
the teams
2:39:52
the teams so not every single microservices is uh
2:39:55
so not every single microservices is uh
2:39:55
so not every single microservices is uh well to say they have all their nuances
2:39:57
well to say they have all their nuances
2:39:57
well to say they have all their nuances and their specialities they're so
2:39:59
and their specialities they're so
2:39:59
and their specialities they're so they can be quite diverse so to
2:40:01
they can be quite diverse so to
2:40:01
they can be quite diverse so to integrate with each of them
2:40:02
integrate with each of them
2:40:02
integrate with each of them um wasn't necessarily plain sailing and
2:40:06
um wasn't necessarily plain sailing and
2:40:06
um wasn't necessarily plain sailing and and then we we could only tap into
2:40:10
and then we we could only tap into
2:40:10
and then we we could only tap into messages that were available at the time
2:40:13
messages that were available at the time
2:40:13
messages that were available at the time when we
2:40:13
when we
2:40:13
when we we created the integration and as i said
2:40:16
we created the integration and as i said
2:40:16
we created the integration and as i said we've got
2:40:17
we've got
2:40:17
we've got um situations where we'd like to look at
2:40:19
um situations where we'd like to look at
2:40:19
um situations where we'd like to look at a large
2:40:20
a large
2:40:20
a large deep volume of data to train a model or
2:40:22
deep volume of data to train a model or
2:40:22
deep volume of data to train a model or do some deep analytics
2:40:24
do some deep analytics
2:40:24
do some deep analytics and so that back data simply wasn't
2:40:26
and so that back data simply wasn't
2:40:26
and so that back data simply wasn't available via
2:40:27
available via
2:40:27
available via uh plugging into the existing
2:40:29
uh plugging into the existing
2:40:29
uh plugging into the existing architecture so it's often an
2:40:31
architecture so it's often an
2:40:31
architecture so it's often an additional piece of engineering to go
2:40:33
additional piece of engineering to go
2:40:33
additional piece of engineering to go and ask
2:40:34
and ask
2:40:34
and ask and coordinate with the source team and
2:40:36
and coordinate with the source team and
2:40:36
and coordinate with the source team and say right we want to get
2:40:37
say right we want to get
2:40:38
say right we want to get the three years of back data out of your
2:40:39
the three years of back data out of your
2:40:39
the three years of back data out of your cosmos db
2:40:41
cosmos db
2:40:41
cosmos db um and that was challenging and a fry
2:40:44
um and that was challenging and a fry
2:40:44
um and that was challenging and a fry away piece of work
2:40:46
away piece of work
2:40:46
away piece of work and then also um
2:40:49
and then also um
2:40:49
and then also um when we had an interruption interruption
2:40:51
when we had an interruption interruption
2:40:52
when we had an interruption interruption to service
2:40:52
to service
2:40:52
to service um on the analytical integrations and
2:40:55
um on the analytical integrations and
2:40:55
um on the analytical integrations and perhaps we
2:40:56
perhaps we
2:40:56
perhaps we missed some messages or lost some
2:40:57
missed some messages or lost some
2:40:57
missed some messages or lost some messages and unable to attain
2:40:59
messages and unable to attain
2:41:00
messages and unable to attain crucial messages we'd have to go back
2:41:01
crucial messages we'd have to go back
2:41:01
crucial messages we'd have to go back and say come you know we've missed these
2:41:03
and say come you know we've missed these
2:41:03
and say come you know we've missed these messages we need them
2:41:04
messages we need them
2:41:04
messages we need them we're missing data and then that would
2:41:07
we're missing data and then that would
2:41:07
we're missing data and then that would also be not a non-trivial exercise to
2:41:09
also be not a non-trivial exercise to
2:41:09
also be not a non-trivial exercise to try and get access to that
2:41:10
try and get access to that
2:41:10
try and get access to that those historical events those historical
2:41:13
those historical events those historical
2:41:13
those historical events those historical messages
2:41:14
messages
2:41:14
messages so before i talk about how we kind of
2:41:16
so before i talk about how we kind of
2:41:16
so before i talk about how we kind of our journey on
2:41:17
our journey on
2:41:18
our journey on on uh addressing some of these cons i
2:41:20
on uh addressing some of these cons i
2:41:20
on uh addressing some of these cons i want to talk a little bit about uh
2:41:22
want to talk a little bit about uh
2:41:22
want to talk a little bit about uh boundaries and ownership so a naive
2:41:25
boundaries and ownership so a naive
2:41:25
boundaries and ownership so a naive approach might be to say well
2:41:27
approach might be to say well
2:41:27
approach might be to say well we you will build some ingestion
2:41:29
we you will build some ingestion
2:41:29
we you will build some ingestion services we'll have to centralize data
2:41:31
services we'll have to centralize data
2:41:31
services we'll have to centralize data team and they'll
2:41:32
team and they'll
2:41:32
team and they'll they'll uh they'll reach into the
2:41:34
they'll uh they'll reach into the
2:41:34
they'll uh they'll reach into the operational stores and
2:41:36
operational stores and
2:41:36
operational stores and and pull off data and as much data they
2:41:39
and pull off data and as much data they
2:41:39
and pull off data and as much data they need the data
2:41:40
need the data
2:41:40
need the data to hydrate data stores for the analytics
2:41:44
to hydrate data stores for the analytics
2:41:44
to hydrate data stores for the analytics functions however this really doesn't
2:41:47
functions however this really doesn't
2:41:47
functions however this really doesn't um separate uh the cons the operational
2:41:50
um separate uh the cons the operational
2:41:50
um separate uh the cons the operational analytical concerns
2:41:51
analytical concerns
2:41:52
analytical concerns and um i can assure you no team at asos
2:41:54
and um i can assure you no team at asos
2:41:54
and um i can assure you no team at asos will
2:41:55
will
2:41:55
will allow uh another domain to kind of
2:41:58
allow uh another domain to kind of
2:41:58
allow uh another domain to kind of simply
2:41:59
simply
2:41:59
simply dip into their operational uh systems
2:42:02
dip into their operational uh systems
2:42:02
dip into their operational uh systems they want to have the control
2:42:03
they want to have the control
2:42:03
they want to have the control and management over those systems so
2:42:07
and management over those systems so
2:42:07
and management over those systems so with this understanding what we learned
2:42:09
with this understanding what we learned
2:42:09
with this understanding what we learned where from where we started from we came
2:42:11
where from where we started from we came
2:42:11
where from where we started from we came up with some
2:42:12
up with some
2:42:12
up with some integration objectives um that we
2:42:15
integration objectives um that we
2:42:16
integration objectives um that we that we'd like to incorporate in a new
2:42:18
that we'd like to incorporate in a new
2:42:18
that we'd like to incorporate in a new approach
2:42:19
approach
2:42:19
approach um so firstly
2:42:22
um so firstly
2:42:22
um so firstly we wanted to access this rich these rich
2:42:25
we wanted to access this rich these rich
2:42:25
we wanted to access this rich these rich sources of data
2:42:26
sources of data
2:42:26
sources of data from an offline state we don't want to
2:42:28
from an offline state we don't want to
2:42:28
from an offline state we don't want to be interacting directly in any way
2:42:31
be interacting directly in any way
2:42:31
be interacting directly in any way with that operational cosmos db because
2:42:34
with that operational cosmos db because
2:42:34
with that operational cosmos db because it's so mission critical we can't take
2:42:36
it's so mission critical we can't take
2:42:36
it's so mission critical we can't take the risk of interruption to service
2:42:39
the risk of interruption to service
2:42:39
the risk of interruption to service we want to be out of replay data from an
2:42:41
we want to be out of replay data from an
2:42:41
we want to be out of replay data from an arbitrary point in time so
2:42:43
arbitrary point in time so
2:42:43
arbitrary point in time so again coming back to that scenario where
2:42:45
again coming back to that scenario where
2:42:45
again coming back to that scenario where we need to go back in time and get
2:42:47
we need to go back in time and get
2:42:47
we need to go back in time and get huge volume today we wanted that to be
2:42:49
huge volume today we wanted that to be
2:42:49
huge volume today we wanted that to be quite a seamless activity not a big
2:42:50
quite a seamless activity not a big
2:42:50
quite a seamless activity not a big engineering
2:42:52
engineering
2:42:52
engineering exercise and then we as i said we
2:42:55
exercise and then we as i said we
2:42:55
exercise and then we as i said we we want large volumes today but we also
2:42:57
we want large volumes today but we also
2:42:57
we want large volumes today but we also want to be able to look at uh
2:42:58
want to be able to look at uh
2:42:58
want to be able to look at uh more real-time signals as i said you
2:43:00
more real-time signals as i said you
2:43:00
more real-time signals as i said you know monitoring the
2:43:01
know monitoring the
2:43:01
know monitoring the the performance of the promotion and
2:43:04
the performance of the promotion and
2:43:04
the performance of the promotion and other signals
2:43:05
other signals
2:43:05
other signals and behaviors of of of what's happening
2:43:07
and behaviors of of of what's happening
2:43:08
and behaviors of of of what's happening now in the business so we want that fast
2:43:09
now in the business so we want that fast
2:43:10
now in the business so we want that fast play as well so we want both
2:43:11
play as well so we want both
2:43:11
play as well so we want both fast and batch access to data and
2:43:14
fast and batch access to data and
2:43:14
fast and batch access to data and developer lander architecture
2:43:18
developer lander architecture
2:43:18
developer lander architecture so that led us to a conceptual model
2:43:22
so that led us to a conceptual model
2:43:22
so that led us to a conceptual model um and before i get into uh into this
2:43:25
um and before i get into uh into this
2:43:25
um and before i get into uh into this slide
2:43:26
slide
2:43:26
slide it may look like there's a ship
2:43:29
it may look like there's a ship
2:43:29
it may look like there's a ship stuck in a very narrow canal um but
2:43:33
stuck in a very narrow canal um but
2:43:33
stuck in a very narrow canal um but that's that wasn't what i was trying to
2:43:34
that's that wasn't what i was trying to
2:43:34
that's that wasn't what i was trying to do and i actually built this slide
2:43:36
do and i actually built this slide
2:43:36
do and i actually built this slide long before the the suis blockage
2:43:39
long before the the suis blockage
2:43:39
long before the the suis blockage um so let me explain so on the left
2:43:41
um so let me explain so on the left
2:43:41
um so let me explain so on the left we've got the
2:43:42
we've got the
2:43:42
we've got the uh the microservices they're very as i
2:43:44
uh the microservices they're very as i
2:43:44
uh the microservices they're very as i said mentioned very diverse
2:43:46
said mentioned very diverse
2:43:46
said mentioned very diverse family and microservices backed by
2:43:48
family and microservices backed by
2:43:48
family and microservices backed by cosmos db
2:43:49
cosmos db
2:43:49
cosmos db and what we want to be able to achieve
2:43:51
and what we want to be able to achieve
2:43:51
and what we want to be able to achieve is that those microservices take it
2:43:54
is that those microservices take it
2:43:54
is that those microservices take it take the responsibility of taking the
2:43:58
take the responsibility of taking the
2:43:58
take the responsibility of taking the the their data and the nuances in their
2:44:00
the their data and the nuances in their
2:44:00
the their data and the nuances in their services
2:44:02
services
2:44:02
services and built packaging that date into boxes
2:44:05
and built packaging that date into boxes
2:44:05
and built packaging that date into boxes into uniforms containers and if
2:44:08
into uniforms containers and if
2:44:08
into uniforms containers and if we can get each one of those to take
2:44:10
we can get each one of those to take
2:44:10
we can get each one of those to take their special data and put it into boxes
2:44:13
their special data and put it into boxes
2:44:13
their special data and put it into boxes the operation of taking it from the
2:44:15
the operation of taking it from the
2:44:15
the operation of taking it from the operational
2:44:16
operational
2:44:16
operational domains to the analytical functions
2:44:18
domains to the analytical functions
2:44:18
domains to the analytical functions becomes very simple it can take box from
2:44:21
becomes very simple it can take box from
2:44:21
becomes very simple it can take box from source a with in destination uh b
2:44:24
source a with in destination uh b
2:44:24
source a with in destination uh b and then for each analytical function we
2:44:27
and then for each analytical function we
2:44:27
and then for each analytical function we open the box up
2:44:28
open the box up
2:44:28
open the box up and we pull out the data we want and use
2:44:30
and we pull out the data we want and use
2:44:30
and we pull out the data we want and use it to drive those functions
2:44:32
it to drive those functions
2:44:32
it to drive those functions so that's conceptually what we want to
2:44:34
so that's conceptually what we want to
2:44:34
so that's conceptually what we want to achieve and then
2:44:36
achieve and then
2:44:36
achieve and then in the next slide i'm going to talk to
2:44:37
in the next slide i'm going to talk to
2:44:37
in the next slide i'm going to talk to you about how we use
2:44:39
you about how we use
2:44:39
you about how we use cosmos was was ended up being quite a
2:44:42
cosmos was was ended up being quite a
2:44:42
cosmos was was ended up being quite a big hero in delivering a
2:44:44
big hero in delivering a
2:44:44
big hero in delivering a sort of a cloud-based solution to that
2:44:46
sort of a cloud-based solution to that
2:44:46
sort of a cloud-based solution to that conceptual um
2:44:47
conceptual um
2:44:47
conceptual um design so i'll just talk through the
2:44:50
design so i'll just talk through the
2:44:50
design so i'll just talk through the architecture here this this
2:44:52
architecture here this this
2:44:52
architecture here this this through this high-level diagram so
2:44:53
through this high-level diagram so
2:44:53
through this high-level diagram so cosmos db on the left there
2:44:55
cosmos db on the left there
2:44:55
cosmos db on the left there um backing our micros mission critical
2:44:58
um backing our micros mission critical
2:44:58
um backing our micros mission critical um
2:44:59
um
2:44:59
um highly available microservices um and
2:45:02
highly available microservices um and
2:45:02
highly available microservices um and then we can access that change feed as
2:45:04
then we can access that change feed as
2:45:04
then we can access that change feed as as amrish mentioned
2:45:05
as amrish mentioned
2:45:05
as amrish mentioned and what we do is we use a an azure
2:45:08
and what we do is we use a an azure
2:45:08
and what we do is we use a an azure function
2:45:09
function
2:45:09
function um to to take a dependency a trigger
2:45:12
um to to take a dependency a trigger
2:45:12
um to to take a dependency a trigger dependency on that change feed
2:45:14
dependency on that change feed
2:45:14
dependency on that change feed and every time a a
2:45:17
and every time a a
2:45:17
and every time a a a a document is created or mutated in
2:45:19
a a document is created or mutated in
2:45:19
a a document is created or mutated in the cosmos db we get
2:45:21
the cosmos db we get
2:45:21
the cosmos db we get we get those messages that fat state
2:45:23
we get those messages that fat state
2:45:23
we get those messages that fat state coming through
2:45:24
coming through
2:45:24
coming through we play that on to the event hub
2:45:28
we play that on to the event hub
2:45:28
we play that on to the event hub and then what that gives us once we've
2:45:30
and then what that gives us once we've
2:45:30
and then what that gives us once we've got those messages flowing through on
2:45:31
got those messages flowing through on
2:45:31
got those messages flowing through on the event hub
2:45:33
the event hub
2:45:33
the event hub our enterprise analytics function can
2:45:35
our enterprise analytics function can
2:45:35
our enterprise analytics function can now say right i've
2:45:36
now say right i've
2:45:36
now say right i've you know i can access this operational
2:45:39
you know i can access this operational
2:45:39
you know i can access this operational data this high value data through
2:45:41
data this high value data through
2:45:41
data this high value data through two channels it's by channel um so in
2:45:44
two channels it's by channel um so in
2:45:44
two channels it's by channel um so in the batch channel we can use
2:45:45
the batch channel we can use
2:45:45
the batch channel we can use event hub change data capture that's
2:45:47
event hub change data capture that's
2:45:47
event hub change data capture that's continuously listening to the event
2:45:49
continuously listening to the event
2:45:49
continuously listening to the event upstream the messages
2:45:51
upstream the messages
2:45:51
upstream the messages and just running it out to blob
2:45:52
and just running it out to blob
2:45:52
and just running it out to blob continuously
2:45:54
continuously
2:45:54
continuously packaging it up put it in a bob
2:45:56
packaging it up put it in a bob
2:45:56
packaging it up put it in a bob container and that's completely
2:45:57
container and that's completely
2:45:57
container and that's completely configurable
2:45:58
configurable
2:45:58
configurable and then when we want to look at these
2:46:00
and then when we want to look at these
2:46:00
and then when we want to look at these sort of more real-time signals and when
2:46:02
sort of more real-time signals and when
2:46:02
sort of more real-time signals and when i
2:46:02
i
2:46:02
i want to know what's happening right now
2:46:04
want to know what's happening right now
2:46:04
want to know what's happening right now in the business we can use stream
2:46:06
in the business we can use stream
2:46:06
in the business we can use stream analytics to also
2:46:07
analytics to also
2:46:07
analytics to also very frictionlessly um hook into that
2:46:10
very frictionlessly um hook into that
2:46:10
very frictionlessly um hook into that event hub
2:46:10
event hub
2:46:10
event hub and that all happens completely away
2:46:13
and that all happens completely away
2:46:13
and that all happens completely away from that operational uh
2:46:14
from that operational uh
2:46:14
from that operational uh cosmos tv and you see that the
2:46:18
cosmos tv and you see that the
2:46:18
cosmos tv and you see that the the main boundaries here are very
2:46:19
the main boundaries here are very
2:46:19
the main boundaries here are very deliberate so we want the
2:46:21
deliberate so we want the
2:46:21
deliberate so we want the uh the source team the team that looks
2:46:23
uh the source team the team that looks
2:46:24
uh the source team the team that looks off that mission critical
2:46:25
off that mission critical
2:46:25
off that mission critical uh microservice to have the management
2:46:28
uh microservice to have the management
2:46:28
uh microservice to have the management control over that azure function have
2:46:30
control over that azure function have
2:46:30
control over that azure function have the management and control over the
2:46:31
the management and control over the
2:46:31
the management and control over the event hubs
2:46:32
event hubs
2:46:32
event hubs one so they can control the throttle the
2:46:35
one so they can control the throttle the
2:46:35
one so they can control the throttle the the
2:46:36
the
2:46:36
the workload on that change feeds they can
2:46:38
workload on that change feeds they can
2:46:38
workload on that change feeds they can they can observe the the pressure on the
2:46:40
they can observe the the pressure on the
2:46:40
they can observe the the pressure on the iu and make decisions about
2:46:42
iu and make decisions about
2:46:42
iu and make decisions about whether they need to upscale or
2:46:43
whether they need to upscale or
2:46:43
whether they need to upscale or downscale or turn off that azure
2:46:46
downscale or turn off that azure
2:46:46
downscale or turn off that azure function
2:46:47
function
2:46:47
function and then they can also manage the event
2:46:49
and then they can also manage the event
2:46:49
and then they can also manage the event hub
2:46:50
hub
2:46:50
hub and who can have access to the data um
2:46:53
and who can have access to the data um
2:46:53
and who can have access to the data um and so you've got the the the access
2:46:56
and so you've got the the the access
2:46:56
and so you've got the the the access control model
2:46:58
control model
2:46:58
control model and i wanted to really highlight here uh
2:47:00
and i wanted to really highlight here uh
2:47:00
and i wanted to really highlight here uh a really
2:47:01
a really
2:47:02
a really neat little um toggle uh that's
2:47:04
neat little um toggle uh that's
2:47:04
neat little um toggle uh that's available
2:47:06
available
2:47:06
available when you create a an azure function
2:47:08
when you create a an azure function
2:47:08
when you create a an azure function that's going to listen to a change feed
2:47:10
that's going to listen to a change feed
2:47:10
that's going to listen to a change feed trigger
2:47:11
trigger
2:47:11
trigger and and you can figure this start from
2:47:15
and and you can figure this start from
2:47:15
and and you can figure this start from from the beginning toggle the default is
2:47:18
from the beginning toggle the default is
2:47:18
from the beginning toggle the default is false it will just start reading
2:47:20
false it will just start reading
2:47:20
false it will just start reading the change feed as as as and when it
2:47:22
the change feed as as as and when it
2:47:22
the change feed as as as and when it starts up and it's live
2:47:23
starts up and it's live
2:47:24
starts up and it's live if you toggle the start from the
2:47:25
if you toggle the start from the
2:47:25
if you toggle the start from the beginning actually you're saying okay
2:47:27
beginning actually you're saying okay
2:47:27
beginning actually you're saying okay when you create your leases to to read
2:47:29
when you create your leases to to read
2:47:29
when you create your leases to to read that change feed
2:47:30
that change feed
2:47:30
that change feed i don't want you to start reading and
2:47:32
i don't want you to start reading and
2:47:32
i don't want you to start reading and reading the change feed now you know
2:47:34
reading the change feed now you know
2:47:34
reading the change feed now you know what's going on now
2:47:35
what's going on now
2:47:35
what's going on now what i want you to do is i want you to
2:47:36
what i want you to do is i want you to
2:47:36
what i want you to do is i want you to start right from the beginning
2:47:38
start right from the beginning
2:47:38
start right from the beginning the earliest um uh event yeah
2:47:42
the earliest um uh event yeah
2:47:42
the earliest um uh event yeah earliest documents in that container
2:47:45
earliest documents in that container
2:47:46
earliest documents in that container and then just play through so if the if
2:47:48
and then just play through so if the if
2:47:48
and then just play through so if the if the cosmos db
2:47:49
the cosmos db
2:47:50
the cosmos db has been there for a year you'll start
2:47:52
has been there for a year you'll start
2:47:52
has been there for a year you'll start with the very first
2:47:53
with the very first
2:47:53
with the very first um record that was created a year ago
2:47:55
um record that was created a year ago
2:47:55
um record that was created a year ago and then just play through and
2:47:56
and then just play through and
2:47:56
and then just play through and eventually it catches up and then you'll
2:47:58
eventually it catches up and then you'll
2:47:58
eventually it catches up and then you'll read in the here and now
2:47:59
read in the here and now
2:47:59
read in the here and now and the nice thing there is that
2:48:00
and the nice thing there is that
2:48:00
and the nice thing there is that actually through one piece of
2:48:02
actually through one piece of
2:48:02
actually through one piece of architecture through one piece of
2:48:03
architecture through one piece of
2:48:03
architecture through one piece of development we get
2:48:04
development we get
2:48:04
development we get we get the two things we're after that
2:48:07
we get the two things we're after that
2:48:07
we get the two things we're after that that volume high volume
2:48:09
that volume high volume
2:48:09
that volume high volume uh high value uh enterprise data
2:48:12
uh high value uh enterprise data
2:48:12
uh high value uh enterprise data um and the what's happening right now
2:48:15
um and the what's happening right now
2:48:15
um and the what's happening right now and they come through the same system
2:48:16
and they come through the same system
2:48:16
and they come through the same system for the same
2:48:17
for the same
2:48:17
for the same piece of uh cloud infrastructure and we
2:48:19
piece of uh cloud infrastructure and we
2:48:19
piece of uh cloud infrastructure and we get our bike
2:48:20
get our bike
2:48:20
get our bike by channel output
2:48:23
by channel output
2:48:23
by channel output um and this has been highly effective
2:48:27
um and this has been highly effective
2:48:27
um and this has been highly effective and successful asos so
2:48:29
and successful asos so
2:48:29
and successful asos so for for example um we processed three
2:48:33
for for example um we processed three
2:48:33
for for example um we processed three years of data from one of our mission
2:48:35
years of data from one of our mission
2:48:35
years of data from one of our mission critical
2:48:36
critical
2:48:36
critical systems in just a few hours with one
2:48:39
systems in just a few hours with one
2:48:39
systems in just a few hours with one engineer sort of managing the the change
2:48:41
engineer sort of managing the the change
2:48:41
engineer sort of managing the the change request and release
2:48:42
request and release
2:48:42
request and release and observing and taking control and
2:48:45
and observing and taking control and
2:48:45
and observing and taking control and making sure that it all flowed through
2:48:46
making sure that it all flowed through
2:48:46
making sure that it all flowed through nicely and there was no introduction to
2:48:48
nicely and there was no introduction to
2:48:48
nicely and there was no introduction to service and
2:48:48
service and
2:48:48
service and you know that's the sort of thing we
2:48:50
you know that's the sort of thing we
2:48:50
you know that's the sort of thing we love uh being able to sort of do that
2:48:52
love uh being able to sort of do that
2:48:52
love uh being able to sort of do that low
2:48:53
low
2:48:53
low low effort um capturing of data
2:48:57
low effort um capturing of data
2:48:57
low effort um capturing of data um and now so i talked about
2:48:59
um and now so i talked about
2:48:59
um and now so i talked about specifically about the the approach
2:49:01
specifically about the the approach
2:49:01
specifically about the the approach in sort of in in a sort of macro level
2:49:04
in sort of in in a sort of macro level
2:49:04
in sort of in in a sort of macro level but i want to talk about now about how
2:49:06
but i want to talk about now about how
2:49:06
but i want to talk about now about how that works at
2:49:07
that works at
2:49:07
that works at scale asos and how we how this
2:49:10
scale asos and how we how this
2:49:10
scale asos and how we how this helps us decentralize some of the uh
2:49:13
helps us decentralize some of the uh
2:49:13
helps us decentralize some of the uh enterprise analytics
2:49:15
enterprise analytics
2:49:15
enterprise analytics um responsibility
2:49:19
um responsibility
2:49:19
um responsibility so here it's very similar to the first
2:49:20
so here it's very similar to the first
2:49:20
so here it's very similar to the first slide earlier in the session so
2:49:23
slide earlier in the session so
2:49:23
slide earlier in the session so uh well for each microservices
2:49:26
uh well for each microservices
2:49:26
uh well for each microservices they adopt this approach they pulling
2:49:29
they adopt this approach they pulling
2:49:29
they adopt this approach they pulling or pushing the data from their cosmos db
2:49:31
or pushing the data from their cosmos db
2:49:31
or pushing the data from their cosmos db to event hub
2:49:32
to event hub
2:49:32
to event hub and then from there the enterprise
2:49:34
and then from there the enterprise
2:49:34
and then from there the enterprise analytics function becomes much simpler
2:49:36
analytics function becomes much simpler
2:49:36
analytics function becomes much simpler and we've got this bioma by channel
2:49:38
and we've got this bioma by channel
2:49:38
and we've got this bioma by channel output from all these rich sources
2:49:40
output from all these rich sources
2:49:40
output from all these rich sources and we can just say right for this usage
2:49:42
and we can just say right for this usage
2:49:42
and we can just say right for this usage we can tap into these microservices and
2:49:44
we can tap into these microservices and
2:49:44
we can tap into these microservices and we can get real-time data
2:49:46
we can get real-time data
2:49:46
we can get real-time data for this use case we can type into this
2:49:48
for this use case we can type into this
2:49:48
for this use case we can type into this uh subset of
2:49:50
uh subset of
2:49:50
uh subset of microservices and hydrate this uh
2:49:53
microservices and hydrate this uh
2:49:53
microservices and hydrate this uh and a deep analytic system on its
2:49:55
and a deep analytic system on its
2:49:55
and a deep analytic system on its machine learning model
2:49:57
machine learning model
2:49:57
machine learning model and what effectively we've done is
2:49:59
and what effectively we've done is
2:49:59
and what effectively we've done is distributed some of that effort some of
2:50:01
distributed some of that effort some of
2:50:01
distributed some of that effort some of that responsibility
2:50:02
that responsibility
2:50:02
that responsibility away from like i guess a stereotypical
2:50:05
away from like i guess a stereotypical
2:50:05
away from like i guess a stereotypical data engineering data analytics team
2:50:08
data engineering data analytics team
2:50:08
data engineering data analytics team and therefore um when we place the
2:50:11
and therefore um when we place the
2:50:11
and therefore um when we place the control and the ownership and the
2:50:13
control and the ownership and the
2:50:13
control and the ownership and the management
2:50:14
management
2:50:14
management in the right place it should it should
2:50:16
in the right place it should it should
2:50:16
in the right place it should it should be with that team that's looking after
2:50:18
be with that team that's looking after
2:50:18
be with that team that's looking after that mission critical
2:50:19
that mission critical
2:50:19
that mission critical uh cosmos db that you know they need to
2:50:22
uh cosmos db that you know they need to
2:50:22
uh cosmos db that you know they need to manage it they need to make sure it's
2:50:23
manage it they need to make sure it's
2:50:23
manage it they need to make sure it's it's highly reliable highly available
2:50:25
it's highly reliable highly available
2:50:25
it's highly reliable highly available and and
2:50:26
and and
2:50:26
and and the uh the function of capturing this
2:50:29
the uh the function of capturing this
2:50:29
the uh the function of capturing this high value data
2:50:30
high value data
2:50:30
high value data doesn't mean an instruction or risk to
2:50:32
doesn't mean an instruction or risk to
2:50:32
doesn't mean an instruction or risk to interruption to service
2:50:35
interruption to service
2:50:35
interruption to service and this sort of takes us on our early
2:50:37
and this sort of takes us on our early
2:50:37
and this sort of takes us on our early steps towards uh
2:50:38
steps towards uh
2:50:38
steps towards uh becoming more of a data mesh orientated
2:50:41
becoming more of a data mesh orientated
2:50:41
becoming more of a data mesh orientated uh enterprise analytics uh
2:50:46
uh enterprise analytics uh
2:50:46
uh enterprise analytics uh department i guess um
2:50:49
department i guess um
2:50:49
department i guess um so yeah that concludes the session today
2:50:52
so yeah that concludes the session today
2:50:52
so yeah that concludes the session today i hope you
2:50:53
i hope you
2:50:53
i hope you uh enjoyed listening in and i think if
2:50:56
uh enjoyed listening in and i think if
2:50:56
uh enjoyed listening in and i think if we got a bit of time we might be able to
2:50:58
we got a bit of time we might be able to
2:50:58
we got a bit of time we might be able to go over any questions that might come in
2:51:09
yes uh thank you so much for um the
2:51:11
yes uh thank you so much for um the
2:51:11
yes uh thank you so much for um the session it was really great
2:51:13
session it was really great
2:51:13
session it was really great being able to walk through um where you
2:51:15
being able to walk through um where you
2:51:15
being able to walk through um where you guys started from the beginning of your
2:51:17
guys started from the beginning of your
2:51:17
guys started from the beginning of your architecture
2:51:17
architecture
2:51:18
architecture and kind of decoupling those services
2:51:20
and kind of decoupling those services
2:51:20
and kind of decoupling those services from the operational
2:51:21
from the operational
2:51:21
from the operational um to and the transactional um sources
2:51:24
um to and the transactional um sources
2:51:24
um to and the transactional um sources i'm sure there's probably um uh less
2:51:28
i'm sure there's probably um uh less
2:51:28
i'm sure there's probably um uh less support abilities tickets coming in
2:51:30
support abilities tickets coming in
2:51:30
support abilities tickets coming in since you guys have
2:51:32
since you guys have
2:51:32
since you guys have kind of given each of your feature teams
2:51:34
kind of given each of your feature teams
2:51:34
kind of given each of your feature teams um the responsibility of ownership
2:51:36
um the responsibility of ownership
2:51:36
um the responsibility of ownership uh for for for that is that right yeah
2:51:39
uh for for for that is that right yeah
2:51:39
uh for for for that is that right yeah that's the idea is to try and
2:51:41
that's the idea is to try and
2:51:41
that's the idea is to try and decentralize some of that for that
2:51:43
decentralize some of that for that
2:51:43
decentralize some of that for that responsibility because
2:51:44
responsibility because
2:51:44
responsibility because the inverse of that is one big team that
2:51:47
the inverse of that is one big team that
2:51:47
the inverse of that is one big team that has to take on all these
2:51:48
has to take on all these
2:51:48
has to take on all these uh relationships and and build all this
2:51:51
uh relationships and and build all this
2:51:51
uh relationships and and build all this uh maybe integrations and
2:51:56
uh maybe integrations and
2:51:56
uh maybe integrations and it just becomes um a funnel and in
2:51:59
it just becomes um a funnel and in
2:51:59
it just becomes um a funnel and in decentralizing we get the scale of
2:52:02
decentralizing we get the scale of
2:52:02
decentralizing we get the scale of effort
2:52:05
thanks awesome uh amrita um
2:52:09
thanks awesome uh amrita um
2:52:09
thanks awesome uh amrita um i i know you mentioned at the start that
2:52:11
i i know you mentioned at the start that
2:52:11
i i know you mentioned at the start that uh change feed was
2:52:13
uh change feed was
2:52:13
uh change feed was game changing uh for you and um gary you
2:52:16
game changing uh for you and um gary you
2:52:16
game changing uh for you and um gary you you took us through a few of the
2:52:17
you took us through a few of the
2:52:17
you took us through a few of the features of change feed i wondered if
2:52:19
features of change feed i wondered if
2:52:19
features of change feed i wondered if you wanted to
2:52:20
you wanted to
2:52:20
you wanted to maybe pinpoint or articulate what it was
2:52:23
maybe pinpoint or articulate what it was
2:52:23
maybe pinpoint or articulate what it was exactly about change feed that was
2:52:24
exactly about change feed that was
2:52:24
exactly about change feed that was was had made some sure an impact for you
2:52:27
was had made some sure an impact for you
2:52:27
was had made some sure an impact for you with what we've been doing
2:52:29
with what we've been doing
2:52:29
with what we've been doing yeah so essentially because it's an
2:52:31
yeah so essentially because it's an
2:52:31
yeah so essentially because it's an event-driven system
2:52:33
event-driven system
2:52:33
event-driven system um we wanted a way to be able to read
2:52:36
um we wanted a way to be able to read
2:52:36
um we wanted a way to be able to read um the log from any point in time and
2:52:40
um the log from any point in time and
2:52:40
um the log from any point in time and the other the other thing is we also
2:52:41
the other the other thing is we also
2:52:41
the other the other thing is we also wanted it to be able to be scalable so
2:52:44
wanted it to be able to be scalable so
2:52:44
wanted it to be able to be scalable so the fact that we've got a a constant uh
2:52:47
the fact that we've got a a constant uh
2:52:47
the fact that we've got a a constant uh changing system with lots of requests
2:52:49
changing system with lots of requests
2:52:49
changing system with lots of requests coming in and the fact that we have a
2:52:51
coming in and the fact that we have a
2:52:51
coming in and the fact that we have a log of all of those different
2:52:53
log of all of those different
2:52:53
log of all of those different um requests coming through meant that we
2:52:55
um requests coming through meant that we
2:52:55
um requests coming through meant that we could actually have competing consumers
2:52:58
could actually have competing consumers
2:52:58
could actually have competing consumers in in two different regions um all
2:53:01
in in two different regions um all
2:53:01
in in two different regions um all uh processing from from a single change
2:53:03
uh processing from from a single change
2:53:03
uh processing from from a single change video if we wanted to or even
2:53:05
video if we wanted to or even
2:53:05
video if we wanted to or even in a single region and that gave us the
2:53:07
in a single region and that gave us the
2:53:07
in a single region and that gave us the scale um that we needed to be able to
2:53:09
scale um that we needed to be able to
2:53:09
scale um that we needed to be able to process all of these things in a timely
2:53:11
process all of these things in a timely
2:53:11
process all of these things in a timely manner
2:53:12
manner
2:53:12
manner and that was one of the most appealing
2:53:13
and that was one of the most appealing
2:53:13
and that was one of the most appealing aspects of this and the fact that you
2:53:15
aspects of this and the fact that you
2:53:15
aspects of this and the fact that you could
2:53:16
could
2:53:16
could um like gary touched on that you could
2:53:18
um like gary touched on that you could
2:53:18
um like gary touched on that you could reset
2:53:19
reset
2:53:19
reset um the pointer back to a previous point
2:53:22
um the pointer back to a previous point
2:53:22
um the pointer back to a previous point in time
2:53:23
in time
2:53:23
in time was an a really good way of us managing
2:53:26
was an a really good way of us managing
2:53:26
was an a really good way of us managing that replay capability that you would
2:53:29
that replay capability that you would
2:53:29
that replay capability that you would see in a
2:53:29
see in a
2:53:29
see in a typical event driven system as well
2:53:35
yeah i guess just to add to that um
2:53:38
yeah i guess just to add to that um
2:53:38
yeah i guess just to add to that um we love the control that you can get
2:53:40
we love the control that you can get
2:53:40
we love the control that you can get with the change feed as i said
2:53:41
with the change feed as i said
2:53:41
with the change feed as i said i think you mentioned you know that
2:53:43
i think you mentioned you know that
2:53:43
i think you mentioned you know that decoupling um so
2:53:45
decoupling um so
2:53:45
decoupling um so yeah maybe we're in a position where um
2:53:49
yeah maybe we're in a position where um
2:53:49
yeah maybe we're in a position where um there's too much demand on that
2:53:50
there's too much demand on that
2:53:50
there's too much demand on that operational system and we can actually
2:53:52
operational system and we can actually
2:53:52
operational system and we can actually just
2:53:52
just
2:53:52
just switch it off okay right well let's just
2:53:54
switch it off okay right well let's just
2:53:54
switch it off okay right well let's just switch it off we don't you know
2:53:56
switch it off we don't you know
2:53:56
switch it off we don't you know analytic feeds is our secondary concern
2:53:59
analytic feeds is our secondary concern
2:53:59
analytic feeds is our secondary concern mission critical
2:54:00
mission critical
2:54:00
mission critical uh availability is our primary concern
2:54:03
uh availability is our primary concern
2:54:03
uh availability is our primary concern and we could and then we can switch it
2:54:05
and we could and then we can switch it
2:54:05
and we could and then we can switch it on later and
2:54:06
on later and
2:54:06
on later and it's kind of like we just pause time we
2:54:08
it's kind of like we just pause time we
2:54:08
it's kind of like we just pause time we you know that's all we do we
2:54:10
you know that's all we do we
2:54:10
you know that's all we do we have to like do a whole healing of the
2:54:12
have to like do a whole healing of the
2:54:12
have to like do a whole healing of the system to switch off okay
2:54:14
system to switch off okay
2:54:14
system to switch off okay we get over that home switch back on and
2:54:16
we get over that home switch back on and
2:54:16
we get over that home switch back on and we replay from where we were
2:54:19
we replay from where we were
2:54:19
we replay from where we were gives us a lot of agility a lot of
2:54:21
gives us a lot of agility a lot of
2:54:21
gives us a lot of agility a lot of control over
2:54:22
control over
2:54:22
control over the um the movement of data between
2:54:25
the um the movement of data between
2:54:25
the um the movement of data between systems
2:54:27
systems
2:54:27
systems going back to the team decentralization
2:54:30
going back to the team decentralization
2:54:30
going back to the team decentralization and how uh you know there's different
2:54:32
and how uh you know there's different
2:54:32
and how uh you know there's different teams working in different sections
2:54:33
teams working in different sections
2:54:33
teams working in different sections about microservices
2:54:34
about microservices
2:54:34
about microservices etc um do you have any tips
2:54:37
etc um do you have any tips
2:54:37
etc um do you have any tips on how to ensure that knowledge transfer
2:54:39
on how to ensure that knowledge transfer
2:54:39
on how to ensure that knowledge transfer is transferred between the team so you
2:54:41
is transferred between the team so you
2:54:41
is transferred between the team so you don't have
2:54:41
don't have
2:54:42
don't have you know two teams working on the same
2:54:43
you know two teams working on the same
2:54:43
you know two teams working on the same thing or if someone wants to you know
2:54:44
thing or if someone wants to you know
2:54:44
thing or if someone wants to you know work on something else they don't have
2:54:46
work on something else they don't have
2:54:46
work on something else they don't have to
2:54:46
to
2:54:46
to relearn everything from scratch sort of
2:54:49
relearn everything from scratch sort of
2:54:49
relearn everything from scratch sort of things
2:54:50
things
2:54:50
things yeah i guess so the main knowledge
2:54:52
yeah i guess so the main knowledge
2:54:52
yeah i guess so the main knowledge transfer is a is a big challenge
2:54:54
transfer is a is a big challenge
2:54:54
transfer is a is a big challenge actually across the enterprise
2:54:56
actually across the enterprise
2:54:56
actually across the enterprise um so you know there's no silver bullet
2:55:00
um so you know there's no silver bullet
2:55:00
um so you know there's no silver bullet to it i mean a lot of it is actually
2:55:02
to it i mean a lot of it is actually
2:55:02
to it i mean a lot of it is actually building um excellent internal
2:55:04
building um excellent internal
2:55:04
building um excellent internal relationships
2:55:05
relationships
2:55:05
relationships and being able to reach out to people
2:55:08
and being able to reach out to people
2:55:08
and being able to reach out to people and
2:55:08
and
2:55:08
and and for people not to feel like they
2:55:10
and for people not to feel like they
2:55:10
and for people not to feel like they can't um make contact or find the right
2:55:12
can't um make contact or find the right
2:55:12
can't um make contact or find the right people to talk to it's all
2:55:14
people to talk to it's all
2:55:14
people to talk to it's all i i described some of our role uh we
2:55:17
i i described some of our role uh we
2:55:17
i i described some of our role uh we might be
2:55:18
might be
2:55:18
might be uh technical architects but i do feel
2:55:20
uh technical architects but i do feel
2:55:20
uh technical architects but i do feel that uh we actually play a role as being
2:55:22
that uh we actually play a role as being
2:55:22
that uh we actually play a role as being cultural architects and try and um
2:55:24
cultural architects and try and um
2:55:24
cultural architects and try and um galvanize those
2:55:25
galvanize those
2:55:25
galvanize those relationships and open those pathways
2:55:27
relationships and open those pathways
2:55:27
relationships and open those pathways and make sure that
2:55:28
and make sure that
2:55:28
and make sure that um that knowledge transfer has is
2:55:32
um that knowledge transfer has is
2:55:32
um that knowledge transfer has is achievable
2:55:33
achievable
2:55:33
achievable with not too much friction
2:55:36
with not too much friction
2:55:36
with not too much friction that makes sense thanks
2:55:43
any other questions that we're seeing uh
2:55:46
any other questions that we're seeing uh
2:55:46
any other questions that we're seeing uh stephanie or theo if not
2:55:47
stephanie or theo if not
2:55:47
stephanie or theo if not uh
2:55:52
all right i think that's it yeah okay
2:55:57
all right i think that's it yeah okay
2:55:57
all right i think that's it yeah okay thank you thank you yeah great
2:56:00
thank you thank you yeah great
2:56:00
thank you thank you yeah great amazing yeah
2:56:04
amazing yeah
2:56:04
amazing yeah that was really good thank you
2:56:11
right so what did we think
2:56:14
right so what did we think
2:56:14
right so what did we think of everything we've just seen um
2:56:18
i mean you know i was really kind of uh
2:56:21
i mean you know i was really kind of uh
2:56:21
i mean you know i was really kind of uh blown away by
2:56:22
blown away by
2:56:22
blown away by how much intricacy there is in what
2:56:24
how much intricacy there is in what
2:56:24
how much intricacy there is in what people are doing out there um
2:56:26
people are doing out there um
2:56:26
people are doing out there um you know they really found creative uh
2:56:29
you know they really found creative uh
2:56:29
you know they really found creative uh solutions to problems and leveraging
2:56:31
solutions to problems and leveraging
2:56:31
solutions to problems and leveraging features in cosmos db in it in a very
2:56:33
features in cosmos db in it in a very
2:56:33
features in cosmos db in it in a very sort of creative and impactful way
2:56:36
sort of creative and impactful way
2:56:36
sort of creative and impactful way especially i mean i'm
2:56:37
especially i mean i'm
2:56:37
especially i mean i'm i'm thinking about um the tooling that
2:56:39
i'm thinking about um the tooling that
2:56:39
i'm thinking about um the tooling that we store at this at the start for him
2:56:41
we store at this at the start for him
2:56:41
we store at this at the start for him for his code and
2:56:42
for his code and
2:56:42
for his code and and the uh um javi's uh uh tooling and
2:56:46
and the uh um javi's uh uh tooling and
2:56:46
and the uh um javi's uh uh tooling and sdk features and so on what do you guys
2:56:49
sdk features and so on what do you guys
2:56:49
sdk features and so on what do you guys think
2:56:50
think
2:56:50
think especially around um the observability
2:56:53
especially around um the observability
2:56:53
especially around um the observability aspect i thought
2:56:53
aspect i thought
2:56:54
aspect i thought that was a really interesting you know
2:56:55
that was a really interesting you know
2:56:56
that was a really interesting you know they they had a need
2:56:57
they they had a need
2:56:57
they they had a need that they had and they found a solution
2:56:58
that they had and they found a solution
2:56:58
that they had and they found a solution that worked for them and
2:57:00
that worked for them and
2:57:00
that worked for them and um they managed to you know improve
2:57:02
um they managed to you know improve
2:57:02
um they managed to you know improve performance and just improve usability
2:57:03
performance and just improve usability
2:57:03
performance and just improve usability in general by
2:57:04
in general by
2:57:04
in general by um and partitioning especially by uh
2:57:08
um and partitioning especially by uh
2:57:08
um and partitioning especially by uh coming up with this tool that worked for
2:57:09
coming up with this tool that worked for
2:57:09
coming up with this tool that worked for them and i can see a lot of like
2:57:12
them and i can see a lot of like
2:57:12
them and i can see a lot of like i guess the words synergies they can you
2:57:14
i guess the words synergies they can you
2:57:14
i guess the words synergies they can you know can be applied across
2:57:16
know can be applied across
2:57:16
know can be applied across um you know many different uh uh
2:57:19
um you know many different uh uh
2:57:19
um you know many different uh uh users um using these kind of things so
2:57:22
users um using these kind of things so
2:57:22
users um using these kind of things so it's great to hear about that
2:57:23
it's great to hear about that
2:57:24
it's great to hear about that yeah just really created a lot of
2:57:25
yeah just really created a lot of
2:57:25
yeah just really created a lot of transparency for the user and and
2:57:27
transparency for the user and and
2:57:27
transparency for the user and and uh a nice little abstraction there um
2:57:30
uh a nice little abstraction there um
2:57:30
uh a nice little abstraction there um for the and these are common things that
2:57:31
for the and these are common things that
2:57:31
for the and these are common things that we see customers doing
2:57:33
we see customers doing
2:57:33
we see customers doing day in day out and they just thought
2:57:34
day in day out and they just thought
2:57:34
day in day out and they just thought about every single one of them and
2:57:36
about every single one of them and
2:57:36
about every single one of them and created a nice little feature
2:57:37
created a nice little feature
2:57:37
created a nice little feature that helps uh so that was cool um
2:57:41
that helps uh so that was cool um
2:57:41
that helps uh so that was cool um and there's there was a lot of theme
2:57:42
and there's there was a lot of theme
2:57:42
and there's there was a lot of theme around change feed uh
2:57:44
around change feed uh
2:57:44
around change feed uh or around streaming so yeah services
2:57:47
or around streaming so yeah services
2:57:47
or around streaming so yeah services you might get the impression that this
2:57:48
you might get the impression that this
2:57:48
you might get the impression that this is a good database for microservices if
2:57:50
is a good database for microservices if
2:57:50
is a good database for microservices if you're watching this
2:57:51
you're watching this
2:57:51
you're watching this and you probably would be right and so
2:57:53
and you probably would be right and so
2:57:53
and you probably would be right and so it just seemed to that we had a lot of
2:57:55
it just seemed to that we had a lot of
2:57:55
it just seemed to that we had a lot of those uh
2:57:56
those uh
2:57:56
those uh conversations and and things coming out
2:58:00
conversations and and things coming out
2:58:00
conversations and and things coming out yeah and um not only that but uh data
2:58:02
yeah and um not only that but uh data
2:58:02
yeah and um not only that but uh data modeling i think that was a
2:58:04
modeling i think that was a
2:58:04
modeling i think that was a big theme yeah the conference is how
2:58:06
big theme yeah the conference is how
2:58:06
big theme yeah the conference is how important actually modeling your data in
2:58:08
important actually modeling your data in
2:58:08
important actually modeling your data in the beginning
2:58:09
the beginning
2:58:10
the beginning is um uh to to the overall integrity of
2:58:13
is um uh to to the overall integrity of
2:58:13
is um uh to to the overall integrity of of your
2:58:14
of your
2:58:14
of your applications because um you'll have to
2:58:16
applications because um you'll have to
2:58:16
applications because um you'll have to then restructure
2:58:17
then restructure
2:58:17
then restructure a lot of things but um it's important to
2:58:19
a lot of things but um it's important to
2:58:19
a lot of things but um it's important to take time to to
2:58:21
take time to to
2:58:21
take time to to to really model your data over time what
2:58:23
to really model your data over time what
2:58:23
to really model your data over time what do you think that this would look like
2:58:24
do you think that this would look like
2:58:24
do you think that this would look like and you know come up with the best
2:58:26
and you know come up with the best
2:58:26
and you know come up with the best strategy moving forward
2:58:28
strategy moving forward
2:58:28
strategy moving forward yeah i really loved pascal's slide i'm
2:58:31
yeah i really loved pascal's slide i'm
2:58:31
yeah i really loved pascal's slide i'm going to seal that slide
2:58:33
going to seal that slide
2:58:33
going to seal that slide it's just so straightforward but this
2:58:35
it's just so straightforward but this
2:58:35
it's just so straightforward but this idea of um
2:58:37
idea of um
2:58:37
idea of um it being a very much application-centric
2:58:39
it being a very much application-centric
2:58:39
it being a very much application-centric process that your your
2:58:40
process that your your
2:58:40
process that your your application design drives your data
2:58:42
application design drives your data
2:58:42
application design drives your data model and then that drives
2:58:43
model and then that drives
2:58:43
model and then that drives what you produce as opposed to data
2:58:45
what you produce as opposed to data
2:58:45
what you produce as opposed to data model and then build your application on
2:58:47
model and then build your application on
2:58:47
model and then build your application on top just a complete mind shift
2:58:49
top just a complete mind shift
2:58:49
top just a complete mind shift and it was just very you know a very
2:58:50
and it was just very you know a very
2:58:50
and it was just very you know a very simple way of
2:58:52
simple way of
2:58:52
simple way of uh representing that and that's what we
2:58:53
uh representing that and that's what we
2:58:53
uh representing that and that's what we see isn't that we see developers
2:58:55
see isn't that we see developers
2:58:55
see isn't that we see developers uh drawn to this this type of
2:58:57
uh drawn to this this type of
2:58:57
uh drawn to this this type of development and and they are much closer
2:58:59
development and and they are much closer
2:58:59
development and and they are much closer to the data model it's just a natural
2:59:01
to the data model it's just a natural
2:59:01
to the data model it's just a natural fit
2:59:02
fit
2:59:02
fit uh it seems like more of a continuum uh
2:59:05
uh it seems like more of a continuum uh
2:59:05
uh it seems like more of a continuum uh as opposed to a kind of let you know
2:59:08
as opposed to a kind of let you know
2:59:08
as opposed to a kind of let you know we'll design the data model and then
2:59:09
we'll design the data model and then
2:59:09
we'll design the data model and then throw it over
2:59:10
throw it over
2:59:10
throw it over um to the developers afterwards so that
2:59:12
um to the developers afterwards so that
2:59:12
um to the developers afterwards so that was a really good way of summing it up
2:59:14
was a really good way of summing it up
2:59:14
was a really good way of summing it up yeah i totally agree i really like that
2:59:17
yeah i totally agree i really like that
2:59:17
yeah i totally agree i really like that slide um
2:59:18
slide um
2:59:18
slide um that really summed it up because there's
2:59:19
that really summed it up because there's
2:59:19
that really summed it up because there's you know there's different groups of
2:59:20
you know there's different groups of
2:59:20
you know there's different groups of people
2:59:21
people
2:59:21
people who um can benefit a lot from cosmos db
2:59:24
who um can benefit a lot from cosmos db
2:59:24
who um can benefit a lot from cosmos db and you know people from the relational
2:59:25
and you know people from the relational
2:59:25
and you know people from the relational world
2:59:25
world
2:59:25
world and people who are just learning about
2:59:27
and people who are just learning about
2:59:27
and people who are just learning about nosql uh we don't have any prior
2:59:29
nosql uh we don't have any prior
2:59:29
nosql uh we don't have any prior database experience
2:59:30
database experience
2:59:30
database experience so it's really cool to see you know the
2:59:31
so it's really cool to see you know the
2:59:31
so it's really cool to see you know the path that they would
2:59:33
path that they would
2:59:33
path that they would um take in either route and you know
2:59:36
um take in either route and you know
2:59:36
um take in either route and you know what's the correct way to do things with
2:59:38
what's the correct way to do things with
2:59:38
what's the correct way to do things with nosql
2:59:39
nosql
2:59:39
nosql um yeah definitely another thing i
2:59:42
um yeah definitely another thing i
2:59:42
um yeah definitely another thing i really enjoyed seeing uh
2:59:44
really enjoyed seeing uh
2:59:44
really enjoyed seeing uh was uh you know how our users are
2:59:46
was uh you know how our users are
2:59:46
was uh you know how our users are utilizing us
2:59:47
utilizing us
2:59:47
utilizing us other azure services to really you know
2:59:50
other azure services to really you know
2:59:50
other azure services to really you know make
2:59:50
make
2:59:50
make their experience as best as it possibly
2:59:52
their experience as best as it possibly
2:59:52
their experience as best as it possibly can be without them having to manage all
2:59:54
can be without them having to manage all
2:59:54
can be without them having to manage all this infrastructure
2:59:56
this infrastructure
2:59:56
this infrastructure um on-premise or on their own on vms
2:59:59
um on-premise or on their own on vms
2:59:59
um on-premise or on their own on vms like they're just doing this in a
3:00:00
like they're just doing this in a
3:00:00
like they're just doing this in a serverless kind of way with azure
3:00:01
serverless kind of way with azure
3:00:01
serverless kind of way with azure functions
3:00:02
functions
3:00:02
functions and connecting that in the three
3:00:03
and connecting that in the three
3:00:03
and connecting that in the three different ways through you know
3:00:05
different ways through you know
3:00:05
different ways through you know measuring changes
3:00:06
measuring changes
3:00:06
measuring changes or through um um you know
3:00:09
or through um um you know
3:00:10
or through um um you know the other the other two ways of doing
3:00:11
the other the other two ways of doing
3:00:11
the other the other two ways of doing that with with uh um
3:00:13
that with with uh um
3:00:13
that with with uh um uh with other functions so that was that
3:00:16
uh with other functions so that was that
3:00:16
uh with other functions so that was that was really interesting to see
3:00:18
was really interesting to see
3:00:18
was really interesting to see yeah awesome um okay so it looks like
3:00:21
yeah awesome um okay so it looks like
3:00:21
yeah awesome um okay so it looks like we're at time so i'm gonna uh
3:00:23
we're at time so i'm gonna uh
3:00:23
we're at time so i'm gonna uh close up here and just again a huge
3:00:25
close up here and just again a huge
3:00:25
close up here and just again a huge thanks to everybody involved
3:00:27
thanks to everybody involved
3:00:27
thanks to everybody involved uh with the with the conference just
3:00:29
uh with the with the conference just
3:00:29
uh with the with the conference just been really awesome and
3:00:30
been really awesome and
3:00:30
been really awesome and and the presentations today were just
3:00:32
and the presentations today were just
3:00:32
and the presentations today were just i'm literally
3:00:33
i'm literally
3:00:33
i'm literally blown away by the quality and what what
3:00:36
blown away by the quality and what what
3:00:36
blown away by the quality and what what people are out there doing
3:00:37
people are out there doing
3:00:37
people are out there doing so just some last things to to say if
3:00:39
so just some last things to to say if
3:00:39
so just some last things to to say if you if this was the first
3:00:41
you if this was the first
3:00:41
you if this was the first uh stream that you watched um there are
3:00:43
uh stream that you watched um there are
3:00:43
uh stream that you watched um there are 18 on-demand sessions
3:00:45
18 on-demand sessions
3:00:45
18 on-demand sessions uh in addition to the live sessions uh
3:00:47
uh in addition to the live sessions uh
3:00:47
uh in addition to the live sessions uh that you can uh
3:00:48
that you can uh
3:00:48
that you can uh get with even more amazing content
3:00:50
get with even more amazing content
3:00:50
get with even more amazing content that's built by our community
3:00:52
that's built by our community
3:00:52
that's built by our community uh if you want to watch those you you
3:00:54
uh if you want to watch those you you
3:00:54
uh if you want to watch those you you just head over to
3:00:55
just head over to
3:00:55
just head over to um aka.ms slash cosmos db
3:00:59
um aka.ms slash cosmos db
3:00:59
um aka.ms slash cosmos db conf hopefully that's appearing in the
3:01:00
conf hopefully that's appearing in the
3:01:00
conf hopefully that's appearing in the banner um and click on the agenda for
3:01:02
banner um and click on the agenda for
3:01:02
banner um and click on the agenda for the event you should you'll
3:01:03
the event you should you'll
3:01:04
the event you should you'll see um what's what's there
3:01:07
see um what's what's there
3:01:07
see um what's what's there another thing would be uh if you've not
3:01:09
another thing would be uh if you've not
3:01:09
another thing would be uh if you've not already join us for our weekly podcast
3:01:11
already join us for our weekly podcast
3:01:11
already join us for our weekly podcast uh so our host and also the mastermind
3:01:14
uh so our host and also the mastermind
3:01:14
uh so our host and also the mastermind behind this whole
3:01:15
behind this whole
3:01:15
behind this whole uh event mark brown uh meets with
3:01:17
uh event mark brown uh meets with
3:01:17
uh event mark brown uh meets with members of our team
3:01:19
members of our team
3:01:19
members of our team as well as members of the cosmos db
3:01:21
as well as members of the cosmos db
3:01:21
as well as members of the cosmos db community
3:01:22
community
3:01:22
community uh to dive into a new topic every week
3:01:24
uh to dive into a new topic every week
3:01:24
uh to dive into a new topic every week so uh if you want to see a
3:01:25
so uh if you want to see a
3:01:26
so uh if you want to see a podcast live or watch it on demand uh
3:01:27
podcast live or watch it on demand uh
3:01:27
podcast live or watch it on demand uh head over to aka
3:01:29
head over to aka
3:01:29
head over to aka dot ms slash cosmos db live tv
3:01:33
dot ms slash cosmos db live tv
3:01:33
dot ms slash cosmos db live tv hopefully that should be coming from the
3:01:34
hopefully that should be coming from the
3:01:34
hopefully that should be coming from the banner as well and finally if you're new
3:01:36
banner as well and finally if you're new
3:01:36
banner as well and finally if you're new to cosmos db this is the first time
3:01:37
to cosmos db this is the first time
3:01:37
to cosmos db this is the first time you've ever heard about it
3:01:39
you've ever heard about it
3:01:39
you've ever heard about it uh and you want to find out ways that
3:01:40
uh and you want to find out ways that
3:01:40
uh and you want to find out ways that you can try out for free
3:01:42
you can try out for free
3:01:42
you can try out for free uh read our blog goes to aka dot ms
3:01:45
uh read our blog goes to aka dot ms
3:01:45
uh read our blog goes to aka dot ms slash cosmos db dash free
3:01:48
slash cosmos db dash free
3:01:48
slash cosmos db dash free and you'll find out four different ways
3:01:50
and you'll find out four different ways
3:01:50
and you'll find out four different ways you can try cosmos db
3:01:52
you can try cosmos db
3:01:52
you can try cosmos db including the free tip tier
3:01:55
including the free tip tier
3:01:55
including the free tip tier where cosmos db will be free for the
3:01:56
where cosmos db will be free for the
3:01:56
where cosmos db will be free for the life of the account that's free
3:01:58
life of the account that's free
3:01:58
life of the account that's free so no excuse
3:02:02
so no excuse
3:02:02
so no excuse so yeah again thanks for joining us for
3:02:04
so yeah again thanks for joining us for
3:02:04
so yeah again thanks for joining us for our very first
3:02:05
our very first
3:02:05
our very first cosmos db conference and we hope to see
3:02:08
cosmos db conference and we hope to see
3:02:08
cosmos db conference and we hope to see you again
3:02:09
you again
3:02:09
you again next year thank you
3:02:12
thank you bye
3:02:22
you

Azure Cosmos DB Conference Session 3

c-sharpcorner.com

Resource boundaries - Kubernetes Fundamentals Ep. 10

Demystifying Azure Container Instance - MVP Show ft. Abhishek

The Cloud Show with Magnus Mårtensson ft. Rik Hepworth Ep. 3

How to use Geospatial Data in SQL Server by Hasan Savran || SQL Server Virtual Conference

Secure Azure Resources using Azure Private Link || IT Security Virtual Conference 2021

The Cloud Show with Magnus Mårtensson ft. Joseph Guadagno - Ep. 9

Developer Best Practices for Azure SQL : Avoiding common pitfalls when using the Cloud Database

The RIGHT Way to Write Unit Tests for Domain-Driven Design applications

Efficient Containerization of .NET Applications using dotnet publish

Salesforce Enhanced Chat v2 Inline Mode in Experience Cloud Site

Smithsonian Starstruck: Immersive VR Experience at Fever Hub in The Florida Mall

Epic Shelf Cloud Rolls Over Georgetown Amid Thunderstorms

Serbia’s Vucic attempts diplomatic ‘balancing act’ with Zelensky’s visit

oren meets world: master memes: social media marketing strategy, content & brand building

Türkiye: Drone footage: Summer meets winter on Türkiye's snow-capped mountains.

Up next in 10

Azure Cosmos DB Conference Session 3

c-sharpcorner.com