Sessão de Dúvidas - Certificação Funcional (07/07/2021)

now what this test has done
already is it has
created a client found the directory
called the token endpoint of the
directory
called the directory to say give me a
software statement and just answer your
question just to see what redirect urls
are supported by
the client you can have a look at what's
been registered in the directory
in the direct the ssl contains all of
the redirect urls so you can use any of
these
but uh in particular you're probably
interested in
um tpp.localhost.callback which is the
one you probably
use so you can use any of these redirect
urls
uh coming back to the certification
suite um so we've registered
we've talked to the bank and
we've discovered the bank we found the
registration endpoint and we've made a
request
to the bank to register and what has the
bank responded with the bank has
responded with
a client that's pretty cool the bank has
has
responded and said yeah you're
registered you can now talk to me and
that's great
does it work that's the million dollar
question and that's the real test of
dynamic client registration there's no
point
registering uh a client and saying
you're done no your client has to be
usable from the moment you click
registration now
i'm just going to repeat that test
because this will look tight that little
time down
but let's go and do that again so i'm
going to talk to the directory you can
see the success count coming up i'm
doing my registration i've registered to
the bank and now i'm going to pretend to
be a new customer
and i'm going to log on and i'm already
logged onto that instance the directory
i'm going to select those two accounts
and i'm going to consent to share my
data
i've redirected to my tpp my tpp is
checking my tokens
and voila i'm done that's it that's how
open banking should be
all right got it uh so so basically
the tpp has should be calling the
directory to
register uh with the bank beforehand
in order to be able to uh
get the redirects and everything that's
correct you've got it
when you're registered to a bank it's
like rocking up with a um
a power of attorney or a signed
registration
letter you you come to the bank saying
here's my
signed registration letter from the
directory and this
signed document contains all of the
information about your application that
you've registered at the directory
but it also contains all of the
uh status about who you are so this is a
entry for open bank in brazil it's
registered and accredited by
banco central it is a dados it
is active and so the bank is processing
all of this information to say yeah you
are allowed to
to on board very simple very
straightforward so
that's our question and that's included
in the sample
uh for the node.js as well
uh this request not yet it will be
it will be by the end of next week i've
got it done i just need to check it in
if you ask me if you send me an email i
can send you some
snippets because it will do that that
that got a
bit confused for me because i was able
to pass
the test from the op id but i didn't do
this step
so that's the only piece i'm missing i
guess to be able to use the
the mock bank right now well you can use
the mock bank using
these credentials because the
credentials are already
on the directory oh and so the probably
the issue was the
the uh forgot the name but the
the thing you showed before that like
there's the
number two and the three and the version
but there's like two
three options for like on the urn
let me check me oh yeah but i think i
got the issue
you shot before yeah well i'll fix the
take the the reference tpp
and uh you'll have an end-to-end example
a lot longer take a couple of minutes
but we don't we're not committing to
keeping this tpp um
accurate now saying that anybody in the
community
can submit a pull request and improve it
so whilst we're giving this to the
community it would be very good if the
community would also give back
there's no there should be no
competition in getting access to data
the competition should be in what you
want to do with the data
what value propositions are you bringing
open banking is not about
creating new pipes open banking is about
creating new propositions
on standards it's not meant to create
new uber
tpps there's no need for them when
you've implemented
uh to a consistent standard and you've
implemented everything
to open standards and so this is a
classic example of how open banking
is meant to function by enabling access
to data
not requiring new pipes
make sense yeah yeah
got it and just to clarify my last point
uh
i i was talking about the acr values
supported
yeah it'll be the acr values that i've
got mixed up
on that yeah so i should be sending
silver right
that's the one no no i will be sending
silver
but you should be sending the acr value
that is published by the bank
so different banks will be publishing
different acr values
um the mock bank supports two or three
of them but
there are two specific standards for
brazil so
when you discover that metadata for each
of the banks you should be using the acr
one of the two acrs that the bank says
it supports
the ones that contain brazil in the name
right the ones that take brazil in the
name
all right you got it every bank is
required to support
loa2 some banks may support loa3 it's
your choice
as a tpp as to which one you ask for
got it thank you no worries that wasn't
what i was
going to touch on today um
uh brownie uh eric does somebody want to
do an introduction as to
the actual purpose of the meeting for
functional performance
excuse me ralph uh uh
ralph yep excuse me
can you can you say it again does
somebody want to do an introduction
to the why we do functional performance
and what the goals are and what we're
trying to achieve
and how it works before i dive into the
details or do you just want me to do it
i'm more than happy either way okay i'll
do in
portuguese very briefly and you and your
make the additional details um
uh
foreign
um
okay ralph uh you can
move on okay so
welcome to how to achieve functional
performance
um it's exceptionally similar
to how you achieve security conformance
what we have created and what we've what
braiding's been doing is working with
the open id foundation
to develop both sides of the tests
so we have in the same way as we
were working to create the appropriate
security and dcr tests with the oidf
we have used the exact same foundation
and framework
to create a set of functional uh tests
for
each of the different apis when you log
on to the service
it's you can log on using your directory
sandbox
accounts at the moment we'll be adding
support
for the directory production accounts as
well
but it looks exactly the same as you did
previously so what you can do is plug in
your details of your client that you
would have attained using dynamic client
registration
so first step register with a bank if
you want to you can prove that that bank
is compliant
for um to the security profile that they
say that they are
and then you take that client and you
add those client details into
the uh the test that you're trying to
perform now
what we've got is we've gone and created
a whole bunch of different
functional tests right at the top open
making brazil functional tests and you
will run through each of these tests
that you think are necessary for
your certification
cases so if you're a bank that supports
personal accounts
only you obviously wouldn't be running
the business account
api you're then going to select the
mechanisms that
you used when you registered with that
bank
and you fill out your certificates put
in your keys blah blah blah blah
everything that we did
said previously you're going to specify
the version number
of the api and this should correspond to
the version or the endpoint that you're
advertising on the directory
to say this is where my accounts api
is in exactly the same way you're going
to use your consents api
plug in your cpf or your cmpj that you
need to create
upfront as part of your consents and you
hit click create test
plan this is all live and we're making
changes so
what we've got here is we've got the
first test which is an account
api test and then we've got an explicit
test for the wrong api permissions
uh test where we explicitly start trying
to put in
wrong data now it's very important for
both tpps and banks that we have
a consistent api and a consistent api
covers both happy path but also negative
path testing which is just as good
now there are a lot of gaps still in the
specifications when it comes to negative
paths
as in error codes or things that should
come back from the authorization
server and there's quite a few new
spec work that's going to go into the
next iteration of these to
standardize more the unhappy and error
states
but what we're starting with is some of
the basic um
basic tests to make sure that banks and
and in the case of relying parties tpps
are correctly handling
bad requests to make sure that they're
not accidentally leaking more data than
the tpp is consented to and that they
respond
correctly if you do something stupid
like put in an expiry
more than uh 12 12 months so
let's just do the negative test now some
of these negative tests are
relatively new um so and this is all
live now i'm making no commitments that
the mock bank handles all of these
but uh we shall we shall see
we've set up the consent and now we're
going to log in with a bad
theoretically a bad test
and i'm going to log in again so i
passed my first test
first part and this time i'm going to
log in
get a second time with a different set
of permissions
lots of tests lots of tests too
so here we go ensure we cannot call the
accounts group api
call protected resource and expect a
failure
i called the expected resource and i got
a failure
so this test explicitly goes and she is
testing to make sure that when i ask for
the wrong permissions
i'm denied access to resources
okay so if we go back to return to my
plan
and i'll run the happy path tests for
accounts
because the test to make sure that we
can always access the right resources
same thing so got some
successes setting up the consent getting
tokens
checking the tokens are correct they're
all meet all the security requirements
um for happy path and i'm going to log
in
same thing i'm going to select two bank
accounts i don't want to have access to
i'm going to click continue
and the test start going through and
testing all of the apis
now if this is set up correctly there
should be one failure
unless it's been accidentally fixed
can everybody see the yeah there we go
can everybody see the little test
counter carry
continue to carry out 187
192 194 there's a lot of different test
cases
so we got an error and we failed um it's
still running so it's still testing and
making you know it wasn't a fatal
failure it can carry on testing other
things to see
what else might have broken but at the
moment it has found one
uh one failure case okay it's finished
it's in a failure state
and very nicely i get my summary and i
can click on that and says right
what was the test the test was
fetch account so something happened on
the batch account
i got a response yeah that was all good
and then it went through and started
looking
at the structure of the account so um
all the different field names were they
there let's come back down to our error
so whatever happened unable to find a
subtype on account
let's have a look and indeed there is no
subtype now subtype is a mandatory
element on the account swagger
and therefore the bank or my mock bank
is not spec
compliant and that is a failure of the
test case therefore i failed my
performance i have to go and fix it
and then come back and re-run my
certification test and suite
any questions
i'm going to ask i'm going to use i just
want to use this as a classic example
so i don't know who the unknown user was
that put this comment in but so excuse
me like translated i just want to make
sure it's the right
that's correct so let's go and have a
look at this question so somebody has
raised a
and i won't say who has raised a
question over the marital status
additional field
and somebody's got some notes somewhere
that said it should be filled in even
though it's an optional field and the
types enum was not running through even
though
there was a null value so we've got a
ticket
out right now outstanding and we'll go
through this how we validate
uh each of these these issues because
it's a really good
um example on how to how we go through
the triage process should you find
something that you don't agree with
so in this case it was a ticket that was
raised
here was the details i'll try and hide
the
person
we received annotations natural person
relationship blah blah blah blah
we received a note in the natural
person's identification validate unable
to find marital status additional info
on the natural person identity api
response
as we verified we are responding marital
status additional info null
so let's go and look at what this means
for when we triage these issues
says this is an optional field and being
filled in is as null well let's check
so based on the swagger which is the
golden source
for all things in the test suite and all
things
in open banking if you look for that
field you'll see that marital status
additional info
is not marked as optional in addition
it's explicitly a regex patent of white
space
and any character something that is not
a uh not null and so
in addition when you start looking at
what it actually means
to support a null in swagger
a null has a very specific
type and the null must be marked
as nullable equals true so if you want
to be able to support or if the swagger
says that you're supposed to support
a null value in a specification
then the swagger needs to be updated to
reflect
that status so marital status additional
info
is not currently nullable uh
under the definitions of the swagger
specification
and that means it is a fail if you use a
null value on that request
now if you'd like to change that
you're not talking to us you're talking
to the functional api team
you can raise a specifications bug and
you can uh
follow it up with uh the specification
you know the specifications team
but all of these nulls where they're not
nulls will fail
the swagger validation which means they
fail the conformance suite validation
it's pretty black and white now wherever
it's not
black and white miro or the central
authority doing testing
may decide to give a conditional pass
but in the case of something where it is
explicitly
called out in the swagger it would be
very hard to justify
people not following it from a different
region because i understand the point
that's being made but
nullable is set on other properties it's
not set on maricopa status additional
income
so coming back to the performance
certification suite
any questions on anything i've shown so
far
no one nothing i think
there is a question here in the chat
yeah
talis uh uh ralph pleased to obtain
functional certification
we must submit all apis to conformance
testing
or we can run the tests only with the
api customers
uh so when you you are required
as far as i understand it under the
legislation to certify
your apis that means that you need to
certify any of the apis
that you support so if we go back to
edited configuration if you support
only personal customers you would only
have to obviously submit personal
customers if you support business
customers you'd have to support
business customers which apis you
support is up to you
that's different business models
different what are different
characteristics but whatever
business model you support or whatever
apis you support the expectations you
support
all of the api so
i don't know if that answers your
question the other thing is
this is self-certification so when you
run this pack
this creates green there is no way for
radium to know
if you are meant to certify all of the
apis
one of the apis none of the apis
that's you making that attestation that
you have done what you need to do
now i would expect that most banks will
need to certify every one of these apis
except
maybe for those that don't support
business accounts
does that make sense tell us
perfect thank you
so now we're going to run the credit
card api and
hopefully this one i don't have an error
in so
you know the form
and the credit card and points
does anyone know what it is at the top
of the heads
now
oh let's do resources one off
thank you frederico
so this time i'm going to point at the
root of my credit card account to v1
accounts i'm going to run a test plan
run my test
it's going to go away set up a credit
card's consent so let's look at the
consent
it's going to oh i've got an error so
straight away i failed
on authorized
remember you may have to remember to
make sure that you save your
configuration for every one of the apis
because your client will be registered
with one token endpoint
uh mechanism so now i'm through
the consent phase i've created a consent
called states i'd push an authorization
and now i'm going to
log on and get consent into the credit
card so
as you can see i've got my credit card
information i'm going to consent to all
the credit card stuff
uh
and it's testing all the credit card api
data it's going through and testing
my credit card list my brand name
my all the same thing one by one and
i've already got a failure so
i didn't get i've got an issue with uh
a product additional info doesn't match
the required pattern
on the credit card api response
i've got a heart failure and the 403
forbidden so i've got a failed test
again on this example here's some
example failures you can go and look at
those at your leisure it's the same
process you go through each of these
different
guys
let's
is
logo on consent access to my resources
lots and lots and lots of resources
tests
now for those of you that are going to
have to run testing
good news is that the program uh
supports selenium
really well and there's examples of how
to drive this in an entirely headless
matter fashion
so for those groups that are looking to
go and automate all of this you can also
use
uh access tokens to be able to go and
drive all of the different tests
so there's a fully fledged api this
which gives you the ability to hook this
into your ci cd
and development pipelines as well so
there's no real there's really no
excuses for
releasing apis that don't pass
functional performance
there's also no reason to go and stand
up an army of testers because
you can hook this into all of your
automation processes which is what i
would
very much uh strongly recommend
so now we're finished and i've got a
pass and everything's good and i've got
my logs and i've looked at it what i'm
going to do
is i'm going to return to my plan and
i'm going to create a certification
package now when you create a service
certification package you've got to
remember that everything that you do
is public and everything is available to
the public scrutiny
so i'm going to click my certification
package i'm going to upload two
two files these are not the open id
foundation certifications they are the
open banking brazil
functional certification terms and
conditions um eric are you on the call
i'm not sure if they've been
communicated yet to the ecosystem
but if they haven't been they will be
shortly
brownie do you know if they've been
published yet
um no ralph um
i'll check with eric here
okay cool so you simply upload your
certification uh document this is
basically a
confirmation that you've read the terms
and conditions in terms of conditions
basically
say this is self-certification i have
done this correctly
these are the terms we if you found to
be to lie
i understand that you can take my
results off we understand that it's
uh that it's public et cetera et cetera
et cetera
and you double click on one of those and
you upload that file the next thing
is yours is the scary one your
certification
of performance now this is not a
certificate that we issue
this is a certificate that you issue
on this certificate is going to be these
details are very similar
who are you what are you certifying
what particular profile are you
certifying what version of the suite did
you use
and what date then on here you make some
attestations
you basically say i have not lied i
haven't i have used
a good implementation i have not made
fake data i have not
hacked my system to pass i have done
everything
that i need in order to create a safe
secure functionally conformant
api and then it needs to be signed and
who it needs to be signed with is a i'm
not sure because that's
inside of the sandbox team in miro
uh but typically it's signed by
whoever did it i.e me when i ran that
test
but also who is accountable so usually
you have two signatures and in the uk
you have the person that ran the program
and then you have the accountable senior
executive who takes full responsibility
to the central bank
that they are confirmed that the bank
has done the right thing
to their best of their knowledge have
done the right thing
there's some additional information at
the bottom that allows you to say
whatever
you know if you want to provide
information about how you you know what
did you do
how does it work what else do you offer
you know but it's entirely
entirely optional in terms of other
stuff that you provide
so we're going to upload the
open banking versions of those files in
portuguese have been
[Music]
released they are on the github
we'll get to that in a second
cool so in this conformance document
you have the equivalent files
uh that's the program
terms and conditions so
your environment that
what you're signing up to who signed it
who's yeah who's asserted very very
similar translating to portuguese and i
assume you've got the other one for
the terms of conditions for open banking
visual functional performance
there you go so you can take both of
those sign them and
update them and upload them into the
certification testing suite
now i've got the tab
so once you've done those uh or uploaded
those two documents you
can prepare certification package and
i've been logged out
this is the curse of live demos because
we've
just done a release it looks like
yep
yeah one minute ago
cool that'll be back up in a minute but
in the meantime what i will do is i'll
switch
to
the same package and the same approach
on the certification
the oidf
pack and if we look at
published results and we look at the one
i've published
a while ago there we go
when you've completed that certification
package your your results are made
public so
anyone can have access to this this
public link
you download the package which will be
just a zip file i'll show you what that
looks like it's a
me
there we go the downloader test
uh a test log that looks like this it's
a zip file
inside the zip file is
a copy uh html page and it's a an
extract of
your test results it's got all of the
stuff that was used to run that test
it's got all of the
results of that test
it's got uh and most importantly they've
each got a signature
signature file so that shows that that
signature was signed by the
testing harness and that means that you
can always correlate to make sure that
this
uh test pack was done on the correct
server it was done on the correct um
you know with the correct appointment
with the correct uh with the correct key
in addition you will always be able to
link
to this
plan id on the public on the server this
is not going to work very well
where you will always have a link back
to the test plan
on the server so you can always go and
explore that you can look at the logs
you can download logs you can go and
explore in detail
uh an individual's certification package
okay now i'd like to hand over to
somebody from
um miro who can walk through how to
upload the certification package to
the
service desk if that's all possible
around
no
is there anyone from the service desk
will be able to demonstrate how that
works
hey bro i'll check here if uh we can
can bring someone to to to demonstrate
this part of the process so there was a
question from
raul about the uh the signing process so
what you had in this example
test log were these two signature
documents now this is not something you
create this is something
the test shell will create when you
download your certification package
and these signed documents are signed by
the directory saying
this came from the sorry signed by the
test shell saying this was signed by the
test shell executed on the test shell
and we can confirm that these this html
page has not been tampered with or
changed which uh
is quite important so that people can
review everybody's logs with uh with
confidence
in terms of signing the pdf again that's
a policy
policy question for miro the open id
foundation takes both digital signatures
and wet signatures in a scanned copy on
that
those pdfs i don't know what has been
set by
sandbox squad so
what round is now trying to do is find
someone that can walk you through the
client view
of how you submit your certification
package
but broadly you log on to the
service desk so the open banking brazil
service desk
and you create a ticket type which is a
request for certification
inside that certification
ticket type will be what are you
certifying
which comments which flavors please
upload your zip file
and when you've uploaded that zip file
it will go through into a workflow
now the workflow is very is relatively
straightforward
the workflow is going to validate
uh that you're all green
it will validate that the test shell
that you ran was
accurate it will validate that it was
ran on a relatively new version
you'll validate that it was run on the
actual server and not on your local uh
your local host
and it will validate that the document
was signed by an appropriately
authorized
individual so an accountable executive
at that point the documents will then be
in terms of publication
everybody's results will be added to
when will be added to the
ralph yeah after you finish um
eric can can uh present
how to do this final step
perfect
in addition to the conformance you'll
end up with a an entry on
the conformance table now this is a
markdown table
brazil will be i believe the website
team will be processing this table and
to put it into a nicer
prettier more colorful version than this
but in the similar vein
to the open id foundations structure you
will have
the name of the organization the
particular sub-brand that you use to run
a link to the performance results a view
of the
html document sorry a link to the
download which will be stored
inside get and then overall
an overall result and you will submit an
individual result
for whatever one of these apis you think
that you need to submit for
okay now this is going to be re-changed
to the particular version so it'll be
version 1.0.0 in terms of those
submissions and as new versions are
published
we will start certifying two different
functional specifications
and in the same way as you've got for
the oidf you will end up with
more and more tables of who certified
what where and why
in terms of discovery this information
will be reflected onto the directory
so the directory will also provide a
means to apis and
via api to tpps to be able to confirm
or to be able to make sure that they're
only interacting with versions of apis
that have been certification
certified if they choose to so for
example
it's possible for a bank to release
version two of a specific specification
but not necessarily a completed
certification of that
api it is therefore very important
that tpps have a way of deciding
do they want to interact with version
one of an api which is certified
or do they want to use version two which
might have more functionality
but has not yet been certified which
means it may have issues
so we're going to add that discovery
metadata and the discovery information
to the directory
so that you can
programmatically determine which
versions of apis you want to interact
with
either certified or uncertified versions
and i think that is enough
so i'll just take the opportunity here
ralph to show the
flow on the service desk side if you
don't mind
i don't mind at all eric over the uma
okay
[Music]
[Music]
[Music]
my
can i go
[Music]
[Music]
m
[Applause]
know
[Music]
uh
foreign
is
is
[Music]
um
[Music]
um
reference
foreign
um
um
m
[Music]
not
foreign
foundation
[Music]
foreign
[Music]
foreign
yes
[Music]
is
[Music]
foreign
this
is
foreign
my security certification can i
do the dcr certification first and then
ask for the flat specification or i have
to do both
at the same time no in fact you can do
dcr first if you look on the website
you'll see the ot foundation
has already certified a couple of people
for dcr
dcr only
okay thank you
[Music]
no worries
oh
[Music]
bye
[Applause]
foreign
my
foundation
uh everyone for being here uh if you
anyone send my passion for us we thank
you for you
thank
thank you
you