Accelerate FastAPI Development with OpenAPI Generator

Speaker 1 [00:06]

Hello everyone, thank you for joining this workshop on Fast API Development with the OpenAPI Generator. To keep it a bit lively, we're going to develop a little web app that will create mystic creatures that will solve real-life problems. But before I dive in, if you want to follow along any of the things that I'm going to do during this workshop, please install the OpenAPI Generator. Can I see maybe a raise of hands for those who already have it installed? Oh, wow, thank you, nice. So for those of you who didn't and you want to follow along, I mean, you don't have to, you can also just look at what I'm doing, but it's going to be roughly, I don't know, more than an hour. So maybe to keep you engaged, it's nice to install the generator. You can go to the repository you see here on the screen. In the readme, there is a very detailed description of what you have to do to install the generator. I will talk now for about 10 minutes, a general introduction, which you don't really need that much. It's interesting, but you don't need it so much for the actual content of this workshop. So please, if you want to install it, do it now. You need to do three things. So you install the OpenAPI Generator. You clone this repository. And you have to create a virtual environment and install some requirements. It's all in the readme. So please go there, copy paste everything, and it should be running. We tested it both on Mac and on Linux. If you have any troubles doing installation, my colleague, Katrine, is standing there. Please raise your hand quickly. Maybe she can help you figure it out. Yes, I think that's it. Now let's go for the short introduction. Can I ask maybe the volunteer to close the door? I will. So where do I work? I work at MALT, and MALT is the leading freelancer platform in Europe. We have around 800,000 freelancers in the pool, which we match with around 40,000 projects. We are 600 MALTers in 15 offices in nine countries. And our main mission is that we provide AI-powered technology that matches the freelancer with the project, maybe not so surprising. I work in the machine learning team. It's around nine members, and it's everything from really core data science researchers to the MO ops, which is me. We have around 10 applications running in production. Our core business for quite a long time was around rankers. So we were focusing, I mean, obvious tasks with the data that we have. So we focused on matching the best freelancers to the project. But this scope broadens as we grew. So we now have also an LLM application that helps the clients write projects. And then we have some smaller machine learning models. For example, a skill matching or a skill extraction algorithm that is based on the Transformer architecture. This is our stack. It's not complete. But it gives a bit of an impression what we work with. So in infrastructure, we deploy all our applications with Docker on Kubernetes using Argos ID. And we deploy it on the Google Cloud. For Azure. So for those just arriving, I go back one slide because maybe you do want to see it. So for those who just arrived and maybe didn't see the introduction, if you want to follow along in the workshop, please go to the repository. It's also in the summary. Please install it if you follow along later. Sorry, that was really short. Yes. So around our stack, so we have, for example, Airflow that daily creates features. And we put it in BigQuery as a sort of cold storage. And we have also Elasticsearch, which is deployed as a feature store. Because we had some troubles at some point that loading all features into memory were taking up so much space. So now we have a feature store that we only fetch the features that we want doing prediction. All our applications are fast API applications, which we'll talk about later. So 10 applications, you need monitoring. We have quite some tools. Maybe sometimes a bit too much. We have Sentry and we have Datadoc, the daily monitor, I don't know, memory usage, etc. All our models are stored on MLflow. And for the LLM, we have, for example, LagFuse where we follow the traces. What will we do today? Well, I'll give you a very, very short introduction into FastAPI, PyDentic, and the OpenAPI specification. And sort of the basics that we need to explain why using an OpenAPI generator is actually going to be so powerful. There are two coding exercises, but just to prove a point why, for example, PyDentic is so cool. Then the beef of this whole workshop is actually having the OpenAPI generator running. And I think when we are around coding five, at this point, everybody has run the OpenAPI generator. So you have like a basic level. Then we go to some advanced topics. So we will look at mustache templating. That is a way to customize the OpenAPI generator. We will look at linting of the contract. And I have one slide on versioning. Then we go and look amid more light topic, I think. We talk about validation. I do a little recap. I'll tell a bit about how you can incorporate the OpenAPI generator inside your CI CD. And at the end, I do a little coding challenge. So an opportunity for you to touch the contract, to work with the OpenAPI generator. I get a feeling maybe a bit more for how this is really working, especially if you at some point are interested in implementing this yourself. Yes. So what is the use case? So I think many of you, or I assume many of you have deployed at some point, an application, a Python application, an API. So we have on the one side, the engineering team. So when you look at, I don't know, REST API literature or something, this is sometimes referred to as the client. And then your engineering team has this Mystic Creature Generator for real-life problems. They have the greatest front end, which I did not spend any time on, as you can see. And there is maybe even a little back end going on there. And on the other side, so this is the server side, there is the Mystic Creature service. And when we use this application, we send the request. And in the request, there's maybe the color, the size, and the problem we need help with. And we send this request. And then our Mystic Creature service returns the solution, which is this lovely unicorn. OK. So far, so good. Let's say now maybe your app is super successful. And you have, you need to make a change to your request body. So maybe your users actually don't want to provide a single color anymore, but they want to provide, for some reason, a list of colors. And then there's also requests maybe coming from your data team, because they say, I improved equity with around 30%, but I really need the number of legs. So you think, you make a change to the API, and you think, ah, the number of legs is probably going to be required. But on the other side, the engineers, they implement this as an optional parameter. Whatever happens, the app is used, it breaks, we have a set unicorn. Slightly different topic, but related. I think you've seen, most of you, the Swagger UI, which is basically an interactive documentation of your API. So you use this documentation for two things. So for visualization, because this is a lot lighter to look at than Python code, for most. And you can interact, which is like the really useful part. So if you would go, we will do that in a minute. If you go to this endpoint and you try it out, then you can send an actual request to your service. And if that is all correct, it will return something. So you see the endpoints at the top. So there's two, a post and a get. And you see that on the bottom there is a schema defined with the request and a response body. So let's try this out. How far are you with who is still working on installing? One person. That's good. And I think when I'm done here, we can start a proper workshop. I'm happy that it worked. So I will go to the instructions. I copy paste. Sorry, I will go where you are. So I activate my environment. I'm in the wrong directory. Yes. And then I'm going to start my application. This actually was the part that stressed me out the most. Not speaking, but doing live coding for 90 minutes in front of an audience. Sorry. I hope you bear with me. Yes. So there we are. Yes. So here is our application. The same one that I just showed you. And let's try, or let's first look at the first endpoint. So this is our Mr. Quitter surface where we send the post to the predictions endpoint. And then we see, and let's try it out. So let's click this. That we can send colors. There's only one color in this example. It's green. And the problem that I have is I want to quit social media. And it should be the critical to be of size 10. So let's try it out. Okay. Our solution is apparently that panic is a force quit of your logic centers. And I now generated a winched pony that has the ability to absorb knowledge from books by touching them. Well, if that won't make me quit social media, what will? Okay, so this works. Let's try something else. So maybe I make a mistake. I do a color 100. What happens? What happens if I run this? Who knows? Which error code do I get? Is someone brave enough to say something? Sorry? Yeah. But yes, that's what happens definitely on the hood. And I will show you. It's a 422. So you get an unprocessable entity. And it actually gives me the warning. It says the input should be a string. So that's actually pretty useful. But why? Why is how does my app know that I put in the integer and how is it doing this validation? So actually, it's not any application. It's a Fast API application. And what is Fast API using under the hood? It's using Pydantic. So you can see here. Yes, I think you can see that. You can see here our endpoint. And we see there is like a post endpoint with the name, the predictions. And then you see there is like a prediction request coming in. And there is a prediction response coming out. And what is this prediction request? Sorry, wrong button. This is actually the Pydantic data model. So we see here the request which defined our color, size, and problem. So the color is a list of, sorry, a list of strings. The size is an integer. The problem is again a string. When we call the endpoint, send the request, Pydantic automatically validates. So it checks if this is actually of the right type. And if not, it throws you an error. Pretty cool. So one more thing I need to talk about. So we saw that Fast API contains the endpoints. And it contains the schema request of the response model that uses Pydantic under the hood to do validation. But you also have this nice swagger UI that we just saw. And where is that coming from? How does Fast API can make this nice swagger UI? Well actually, it's doing that from OpenAPI specification. I don't know if you ever tried it. I didn't before I worked on this project. You can go here and you can press OpenAPI JSON. It doesn't look so nice. Let's do the pretty print. And here we see the blueprint of our whole application. So you see there is some general information like the title, the version, and then we see our path, which is the predictions post path. And then you see, for example, here. Do you see my mouse? Yes. Nice. So you can see here there is a schema returned of the form prediction request. And ah, so what does that say? Our prediction request should be of this, you know, those three properties that we use as the color, the size, and the problem. Okay, so now we have all the ingredients. Let's circle back to the initial problem statement. So what we did in the past when we wanted to make a change to our API, let's say adding those number of legs, we would discuss this over Slack or in a Yira ticket or something like this. So those changes were not formalized, but also they led to issues because someone thought maybe it was required, the other thought it was optimal. So what we did is that we made those changes directly to the endpoint, to the data models, and sort of hope that we discussed everything correctly beforehand. And then our backend team actually used the OpenAPI specification that Jason had just showed you. And we gave it to them. Let's say if you want to talk to your API, look at this. We think it works. What turned out is that somehow this OpenAPI specification that was generated by FastAPI was not standard enough to work with our Kotlin backend. Most of the time, like in some cases. But enough to, yeah, cause us some inconvenience. So we were living in this sort of code-first world where we made changes and then see what happens. Another thing which was not really an issue, but not so nice if you're growing as a company. So we had 10 applications running and each developer had a bit of his own style, a bit of his own preferences. So our apps look slightly different. So then we changed to a design or a contract-first approach. What does that mean? You actually take the OpenAPI specification, so this JSON file that I just showed. You use the OpenAPI generator, which we'll run in a minute. And from there, you generate basically the whole application. So you generate your FastAPI app. You generate the endpoints, the data models, et cetera. But also on the client side, sorry, on the engineering side, the client side, they also take the same OpenAPI specification and generate our client and data models from there. If this is slightly abstract, you can also think of this as maybe another Python application that uses your app, and then the client is in the form of a library or something. So, yes. So I think we are now ready to run the OpenAPI generator. So I hope everybody has installed everything. It would be preferable if you check out the main workshop branch, because sort of the main contains sort of the answers. It doesn't matter so much, but I think it's nice if you check out this branch. And then we will run a generator. And I will tell you first what we actually put in to run the generator. You can copy all the commands, by the way, from instructions MD. Please do that also, because some of the commands I show you here are not complete. This one, for example, is not complete because it's way too long. Depending on your installation also, you should either have this CLI there. I think if you have installed the OpenAPI generator with brew, it's without the CLI at the end. But you can see it in instructions, both commands are copied and just copy-paste the right one. Yes. So we will now generate our endpoints. We put in a couple of things. So the first thing is the contract. So the contract is the OpenAPI specification that I just showed you. I will show it again, because we don't use this JSON. We have made a Yamaha file out of it, because it's a little bit better readable. So we put in the contract. We tell the generator which generator we're going to use. In this case, it's going to be a FastAPI application. There's also a Flask. I think there are lots, like the repository is huge. Also for Java, you have tons of generators. You can specify templates. I will come to that later. But you can use this to customize the vanilla implementation of the generator. And then there is an ignore list, which is huge, because it generates Docker files, a README, requirements, txt, a toml file, et cetera, like tons of things. And I'm only interested in actually generating the base FastAPI application, so the main, and the data models, the pidentic data models. And then I put one additional property, the source folder, which is the creature surface. The default name is something less descriptive. I like this better. There's no real reason. So I will go to the instructions. Yes, I will turn this off. Yes. So we are now in the main workshop branch. You can see here the contract. So it looks very similar to the JSON file I just showed you. There's some general info. And you can see, for example, the path. So this is what we put into the generator. And then you can see that this, in the open API server, you see that the APS folder is empty. The model's font is empty. There's also no main.py. So let's run the generator. I copy paste that. Sorry. Yes. Here we go. So a lot of things appear on the screen. Maybe we can have a brief look. You see some files that are ignored, which we didn't care for. So apparently also a lot of init files I removed. And you see, for example, it's writing creature surface main.py. If something goes wrong at some point, you can see also here in this output, maybe a hint of how you can fix that. Yes. Then you can already see in red that I created a lot of files. So I created the API in main. And I created a model. So let's first look at main. Is this big enough for everyone, by the way, in the back? Maybe you can. Yeah. Thank you. So this looks familiar. So this app is a fast API application with the title, description, and the version. And then it says app include router. So let's see where that is coming from. So it's coming from this folder. And let's have a look. Ah. OK. So you see that actually here to this router are attached the endpoint that we need. So the predictions endpoint. There is a prediction response returned. And if you look here, there's also a prediction request. OK. So that looks actually pretty nice. Now what is returned? We look into the prediction request later. But yeah, not going to be surprised. It's going to be, but I think data models we already saw. What is returned here? So this is a bit, I think, like the whole gist of the whole OpenAPI generator. You see that there is a base creature service generated. So let's look into that one. And this is a class with basically an empty skeleton. So there is like here. Sorry. I should not move my mouse here. So this is a stub. And these three dots basically tell you that you should write the implementation elsewhere. And what is this implementation? Well, this is your actual surface, like the actual functionality of your application. So how do you do that? So it tells you here somewhere that it imports from implementation. So this is the implementation folder you see here. We already did that part for you because we're not going to concern ourselves so much with the workings of the API. We're also only interested now in generating the whole application. So you basically take the same, I don't know, skeleton as we saw on this stub. And then instead of returning nothing, you now return the actual response that comes out of the surface. So that's it. The only thing you have to be careful about is that when you rerun the generator again, is that everything that's on the APs, everything that is on the models will be overwritten. So you can only write your own part, like, or have to put it in a different name. But everything that is, like, taken care of by the OpenAPI generator will be overwritten if you write it again. So maybe let's see if it's actually working. No, I wanted to show you one more time that also the PyDentic data models work. So sorry, maybe I should go here. So just you see that we return here for example, that we put in a prediction request in this endpoint. And when you go into models and you look at prediction request, you see that indeed this is the PyDentic data model that does the foundation for us. So this is exactly what we want. So all this coding around models, around the API is generated for you. No Gen AI, always the same, no hallucinations. Pretty nice. Yes. Then I wanted to start the application to see if it's still running. I did not choose the optimal folder structure. I'm sorry. I didn't realize that only when I started rehearsing the talk and I didn't fix it. So we switch back and forth between the root and the creature service itself. I'm sorry for that. No, I wanted to do, I wanted to start the application. Yes, here we go. I think I had it here. Yes. So let's refresh. Let's try out our post one more time to see if it's still working. And it is. Yes. Maybe look at the other endpoint and then I tell you a little bit around the implementation. So let's try this one out. So this is a simple get endpoint where I just get a creature based on its ID. Nothing special there. Let's try it out. So I get now my prophetic stingray, sorry, that can juggle in tangible concepts. Wow, not too bad. Maybe someone has a problem? No? Yeah? No? No? No, but I mean more problem that you want to test the creature service with. Does someone there? No one. No one. No one. Get rid of nightmares. Get rid of nightmares. Thank you. I will let's... Did it work? Wait, that's right. Yes, sorry. Well, apparently the solution is vegetables are just medicine that tastes like dirt. Eat them anyway. So maybe this creature that creates edible clouds will help you get rid of the nightmares. So a little bit on the implementation itself. It's a very, very, very simple... Wait, I'll show you using the repository. So there is a database where you define some creatures, where you define some solutions and does some very simple NLP parsing. Taking a word from the problem and then matching that with the creature. You know, in real life you would probably have like a real database and maybe you have like a proper generator or something else entirely. All right. So I talked about this. Let's look a bit more at the contract because if you're new to that, yeah, YAML files are, I don't know, maybe not always so nicely to read as maybe Python stuff. So you see all the way at the top, the OpenAPI 3.1.0. This is actually not a version of the OpenAPI generator. This is the version of the OpenAPI specification. So this defines how this contract can actually look like. It defines how you can do inheritance, what kind of data types are possible, et cetera. It shows two major versions, III and II. I think one of the things you can do in III that you couldn't do in version II was you can use this component, so reusable schemas that will help you, yeah, I don't know, organize your code in the contract a bit better. Then you see Info. This is like the, I don't know, what you see on the top in the Swagger UI. So this is your own application. And this is also the version that you can change. So this is the version of this contract. And then you see the path, so the different endpoints. It will give you like a, you can say that it's a post or a get. And then you define, well, you should at least define probably your 200, like your OK response, and the schema. So we saw this already. This is the prediction request and response. So let's try to make a change. We start with something pretty straightforward. We're going to change in request the color from green to purple. So let's try it out. Yes. So you can see here in the prediction request, we have colors defined. And we have an example. And the example was purple. And we're going to change that to green. So what do I do? I run OpenAPI generator again. It regenerates everything right, not only the file that you've changed. But let's have a look in the request. And what do we see? Yeah. New color. Yeah. OK. New color. Let's maybe try something more advanced. And let's say we want to add another endpoint. So the use case can be maybe as follows. So maybe your data team has deployed or has an improvement of that creature generation. But you don't want to open that up to, I don't know, the whole company or to all your new clients. So you just create a second endpoint. So the request is the same. The response is the same. You only plug in at some point a different model in the back. So how do you do that? Well, basically, you can just copy paste everything that is under what was your initial endpoint. And I just put a few tools because that is the process to make a second version of your endpoint. So let's copy this. Let's put it in the contract under the first prediction endpoint. Let's press the right button here. I have passed twice. I should remove that. This looks fine. Let's see if it actually works. This looks good. Now let's look into the API. We see predictions. And we see prediction fee too. So maybe now this already gives you, because changing a color is like, I don't know, you could have also done that, I think, quite easily when you have like your not contract first approach, but you can just change something in the Pedantic Made model and just change the example. But maybe here you see a bit more of the advantage of using the generator, because you have in the contract everything centralized. And you just take there, I don't know, you create another endpoint with a different name, but it reuses all the schemas, et cetera. You generate it again. So all those codes around the API is already there. So let's see if it works. Sorry it was here. Yes. There it is. So we have a fee too. Is it actually working? Let's try it out. Yes. Why? Well, I already anticipated that this was going to happen. So already in the implementation that we wrote in the repository, already put in the fee tool. So it was just not used in the first place. Yes. Okay. What did we do so far? So we showed you a little bit how to write a contract. On the contract we run the generator that generates the APIs and the endpoint that creates the podentic data models for you. And the only thing that you do is that you add the code under the implementation. You may think at that point, how am I going to start writing this contract? How am I going to write this first blob? What we did, so we already had apps running in production and we took just open API specification and then we started from there and just try to run the generator and see what happens. If you don't have that, maybe there's a similar application that you can use. Maybe you can look on the web. Maybe you can ask Cloud. I think nowadays if I would do that or what I will do maybe going forward is actually to have Cloud write those contracts for me. Because you know, they're very standardized. You can add like, I don't know, other kind of returns like 404s, etc. Yeah. I think this is something that could be nicely extended. You also think now, okay, but how is the generation part really working? You know, how does it take only the contract and it generates everything from me? Well, it uses something called mustache templating. It's a bit small, I think, but I hope you can read it. Sorry for that. So this is the contract. And then this is the template. This is the mustache template for the main .py. You can already see, I know in the middle, there is this title, app name, description, version, etc. So what actually happens when you run a generator is that it just takes that information from the contract and puts it in. And then you have your first API application. I mean, there's some Java involved in the whole generator itself, but this is like the logic of how that works. There are basically templates for each file that you're generating. But of course, when you look, for example, at the data, but in the data models, there's a lot of reusing of the same kind of mush test templates. And why is that useful to know? Well, the first one is maybe you want to extend the template because you have slightly different needs than the vanilla implementation. So let's try that. So maybe you want to add the prefix unicorn in front of all your endpoints. So how do you do that? So what we did for you is that we took two templates. So one for main and one for the other example, and we put it already in the repository. But if you want to see all templates, you have to run this little command. I put it here because it's not super clear on the website or the documentation from the OpenAPI generator. So let's try to add unicorns everywhere. So what you do is that you do it like this. How do you know how to make these changes into the mush test template? You have to read it. I don't want to do that. But if you don't want to do that, what I did in such cases is that you just read the FastAPI documentation. Right? So if you have a need like adding this prefix, you read the documentation, where do I need to do it? Then you need to find the template that goes with this implementation and then you can add it. So this is quite a simple example. Maybe you had a prefix. You can look this up at the FastAPI website and then you can adjust the template. Let's see if it works. So let's stop this. And let's start the application. Let's refresh. And ta-da! We have unicorns everywhere. So that worked. What is the other example? Well, the FastAPI generator, unfortunately, is still in beta, which means it's from time to time a bit buggy. And one of the most annoying things that we found is that it doesn't generate the examples that you put in your contract. It doesn't put them in your Swagger UI, which is quite annoying, right? I mean, in this case, if you don't see the examples anymore of the color, of the size, you can think, okay, you know, I can think of something myself. But if you have like an actual environment, for example, an integration environment where you're matching freelancer to project. And each time you want to test it, you have to have to look in your database and find the project ID and copy paste that to your Swagger UI. It's a bit annoying. You don't want to do it over and over again. So it's really nice that you can put in examples that actually work to test your application. At least for me, like most of the initial bugs that I find is just by running this first initial try it out function on my application. So this is the general example. This is the first general example of the PyDentic data models. This what you see here under examples is mush test templating. It doesn't look light. It's something that I wrote. It's maybe not the perfect implementation of mush test templating. You are very free to reuse it, but it's quite cumbersome. Let's remove this implementation and then see what happens. So what the implementation actually does is... So this is the standard implementation of the OpenAPI generator. Let's see what it does. Wait. I have to go up. Sorry. Let's have a look on this request. We can already see it from here. You see... Well, no surprises there, right? There are no more examples generated. So let's have a look at it one more time from the UI. So gone are the examples. This is annoying, right? This is something you really don't want. So we fixed that by adjusting the template. As I said, you're very free to use that. If you find a better solution or a better templating style, please let me know. The obvious solution would be also maybe to make now an actual pull request to the OpenAPI generator repository and actually fix that in the generator itself. So we already created an issue for version 7.10. We're now 7.20. It still persists. Yeah. I mean, it's an open source tool, but realize that there are things that are not perfect. And this was our biggest pain point. So, then another topic. Is the contract always correct? And what do I mean by that? And I will show you that by doing an example. Yes. So, we look again into the request. That's my favorite schema of the day. You see that we have here three required parameters in the request object. So we give it colors, size, and a problem. And they're all three required. But maybe, it's the end of the day, I'm a bit tired, I make a typo. I write colors. It could happen. So what happens? Will the OpenAPI generator still run? No. Someone says no. Let's try it. It does. Let's look at the request. It's optional. So the OpenAPI generator does not check the correctness of what you write. I don't think it's a bug. It's something, you know, it only checks if it's like correct YAML. It doesn't go in and say, ah, you specified a parameter that doesn't actually exist. But this is annoying because we made typos. Let's look at one other example. What did they have in mind? Yes. Let's look at another example where the example and the data type don't match up. It's again in the request my favorite schema of the day. So let's say I put here by accident a string. But the example is still 10. If I run the generator now, will it work? Probably yes. Probably yes. So let's try. And let's have a look at the example. Ah. There is no example because I just said the mush test templating. Let's undo this so we can see the example. And let's look again. And then we see here size with a 10. Really difficult to find also. You know, 10 is now a string. It's correct Python. Difficult. What will happen now if I start the application? Will that still work? Let's have a look. Someone wants to take a guess? You have to think a bit about serialization and decen- No, I see a no. Um... No. This breaks. But anyway, still annoying. Hard to detect. What do you do in this case? Um... So as I said, I don't feel this is like a bug or something in the OpenAPI generator. This is just us making like a type and the OpenAPI generator only cares about if it's correct YAML. It doesn't go and look into every bit of like the content that you actually write. Um... How we solve that is that we use another tool. Because apparently this is something that more people struggle with. You can install it if you want. You don't need to. I will just show you how it works. Um... We compared a few and this was the best. Maybe there are newer tools actually on the market. Um... This is from one and a half years ago. So we use something like Spectral. And this is like advanced linting. And that will actually check the correctness of your contract. So I will show you what it does once you have that installed. Let's copy it. Let's stop the application. Am I in the right folder? No. Yes. So it gives us a couple of warnings. Which is nice actually. You can also have a look at those. Sometimes it points you to components that you no longer use. Because you put something in the contract and you didn't update it correctly. Um... Apparently there's warnings that there is no contact in the info. Which is fine. And that gives me an error. But only one. Unfortunately. So it tells us that the example type and the string don't match. You know, we actually want it to be in the chair. But this gives you like a hint to check in the contract if your data type and your examples actually match up. The other thing that we had, that we had a required parameter doesn't exist. So you can extend Spectral by adding your own functions. You have to write it in Java. But you can add those functions. So we have a couple of them. We have required parameters should all exist. Defaults. Required parameters cannot have defaults. And it was one other. I forgot it. But you can add those rules yourself. And then you have like more security that your contract is also really not only, I don't know, correct YAML but also makes sense. Yes. A very short slide about versioning. So we don't really open up our APIs to outside clients. So we don't have a super strict versioning method. So what I just showed you, if you add an endpoint, which is very simpler, but you would only open the time to download it. So if you add a template to the template, it's not quite easy to just copy-paste the original version of this endpoint. Add it. Have a V2 or V2.1 etc. and just regenerate it from there. So don't touch the open API version because that is the version of the open API specification. And the only thing that we ever change is this version that's under the info, which is the version of the contract. And we do this major minor patch logic. So patch is for bug fixes. So maybe there's a typo in one of the examples. The minor is maybe if you change a required parameter to be optional. So that's still backward compatible with older versions and major for any breaking changes. Yeah, let's look at one of the last features of some features that you can add to your contract. So we looked at like what is in the contract now are sort of like the default data types, like string, integers, etc. But maybe you want to do more advanced things. And this is also still capable with the open API generator. So you can add quite straightforward a min and a max length to your string. I don't know, maybe this is useful if you have an LLM and you don't want to have your users to use and send infinite text, but also have like enough text there to give you enough context to answer the question. So let's do that. So let's do that. Let's try it out. Here is the contract. Let's add those min and max length. Sorry. Then we generate the contract again. Let's look in my favorite schema. And we see there is now actually a min, sorry, a min length and a max length defined. You can do a bit more advanced stuff. You can also say maybe I don't want to have just like the min and the max length. I also want to make sure that they only use numerical characters and no, sorry, no numerical characters for some reason. What happens then? That is actually a bit more exciting maybe. You add this rex pattern, sorry. Let's regenerate. And then we see that there is a field validator added, sorry, that matches this pattern. So let's try it out. Sorry, it's regenerated. Let's start the application again and see if our changes worked. It was here. Yes. Sorry, I'm refreshing this to be sure. Let's try it out. So let's try to just want something. No, I can't because I have to have at least 10 characters. And then does the pattern thingy also work? No, I cannot use cool language. So those two things work. There is a lot more you can do at validation of any kind. Also doesn't always work perfectly. Unfortunately, unfortunately, those patterns thingy, I forgot exactly. I could look it up if you're really interested. I think if you had a list or something with a pattern that also didn't work. So don't give up. This is sometimes also the OpenAPI generator not functioning perfectly yet. I want to add a few words on CI CD before we close off the main part of the workshop. So maybe you're not convinced and you say, OK, let's convert everything to the OpenAPI generator. So what do I do when I deploy this application? So I have an implementation that I wrote myself. I have this contract that I wrote myself. But everything that was generated with the OpenAPI generator, should I check that in my repository? Should I let it be generated during the CI CD phase? So if I let it only be generated during the CI CD phase, I can locally also generate the start the app. And then there's a sort of, I don't know, that doesn't for us or as a Python developer, it doesn't feel nice to not see all the files that you're working with and actually start the application locally. So you can do it like this. You can also have the, but then you have maybe this issue. So, OK, you check also in the generated files that allows me to start the app locally and to see all the data models, et cetera. So I have a more better understanding of how my app functions. And also when I make a small change, I see this also directly reflected in the data models, which is sort of nice. But then you could have the case that doing debugging, someone accidentally makes a change in those data models or in that API, right? And at this moment, you start to deviate the contract from the API, which is exactly the whole thing you wanted to avoid. So what we do is that we have the user created contract. We generate the whole API. So we have the endpoints, we have the models, but also during our CI CD pipeline, we also generate the API again. And so all the endpoints, all the data models, we compare them and only if they're equal, it's deployed. So during the CI CD pipeline, when it runs, there's a little Docker image pools with the API generator that then generates the API from this contract. So what do we do so far? We wrote a contract. Well, I mean, it was available. We ran the generator multiple times, making changes to the contract that generated the API, the endpoints and the data models. And we added the code like the beef of the whole application and implication. I showed you how to do some, like, I don't know, first version validation. And then I gave you some pointers how you can actually extract the templates and customize the generator to your own needs. Then if you want to make sure there's still contract compliance, so the contract and APIs and the endpoint are not going to deviate. We have this solution in our CI CD pipeline where we actually check that the generated files are still the same, are still matching the contract. And I also showed you a bit of inconvenience. You know, it's always the case that you have a contract that is correct YAML but it's not correct and correct contract. So think of this, you know, required parameters didn't exist. And therefore you can use like an additional tool like have a little, have a linter. Yeah, so I hope you enjoyed and learned something. Maybe it's time to play a bit with the contract and a generator yourself if you'd like to. So my first question, maybe an easy one, is I have these three required parameters, again in my favorite schema. So I have color, size and problems and I would like to make size optional. How do I do that? And don't say make a typo. It's not a trick question. How could I make one of these parameters optional? Yes. Yes, yes, exactly. We just remove it, right? When you saw it, when I made a typo, it already became optional. So you just remove it and then it's no longer required. So by default, the default state is optional. So this is a bit more advanced. And this is something so for all these kind of things you want to do, like the Swagger UI has the documentation of the OpenAPI specification. You can look there and you can find almost everything. It's not almost 100% possible within like 90% works within the OpenAPI generator, not everything. That's just because it's still in beta. So the second challenge is let's try to restrict a property to a list of fixed values. It's a bit more challenging, this one. There is a hint on the slide. Tell someone, I give you a bit of time. So what about enums? Something about enums. That's definitely the way to go. Maybe I go here. Does it work? Yeah, nice. I zoom in a little bit. This allows you basically to restrict a list, for example, to a specific or like a list or a string to a specific subset. So let's try that in our case. Let me see if I can code this directly correctly. No. Sorry. Colors. Sorry. I just want to make sure I'm changing the right property. Yes. And I think it should go here. Let's try it. If it doesn't work, it doesn't work. There is an answer somewhere in the repository on the main branch. Maybe I shouldn't tell you that already. Let's try again. This worked. Let's look in the request. Ah. So nice. We have an additional field validator which actually says the color should be black, white, red, green, or blue. So let's try. Let's just try it out. Let's see what happens if we do the color purple, for example. No. So, that works as well. So, that's pretty nice. Let's look at the final example. Yes. Yes. And let's start the application. It was here. It was here. All righty. Let's try it out. Maybe we try with the color pink. No. So, that's pretty nice. Let's look at the final example. It's around inheritance. In the prediction response, a new schema. Let's say that instead of returning the Mystic Creature alone, I would like to inherit all the properties that are in the Mystic Creature, and then add four legs. You can look again at the Swagger documentation and, yes, maybe you have an idea, there's a little hint already on the screen, how you could do that. Can you speak a bit louder? I think that was correct. Yes, yes, okay. So this is indeed the right solution. I'm going to, I think, copy-paste that, I'm sorry, from my main branch to make sure. Now let's try it. Maybe let's just try. Let's see how far we get. I think we have some time. So in the response, where are you? Here we go. There is no longer, I'm not returning a creature. I'm not returning Mystic Creature anymore, but a creature. Ah, look, it gives me already some example. Which is all of, which is this one. This one. And then I'm going to add another object. properties legs, integer, example, four. Let's see if this works. No, I made a mistake. So I'm going to check the answer because I'm not so good, I think, in live coding in front of this huge audience. Sorry about that. Where is it? Legs type, maybe I made a typo here. Type objects, properties, am I having the indentation wrong? Legs. Legs type indentation. Thank you. You're better than me. Wait, I lost it. Here, here, here. Thank you. Let's try this again. Yes, this looks better. Let's look at the response. We see actually we also have now an additional, we have like now a response creature. And this response creature, so the prediction response, maybe I should show this first. So this is like a creature, no longer a mystic creature. And there is now a prediction response creature that contains the legs. So pretty nice. So you can do inheritance also within this contract. Yeah. I think we're a bit quicker, but maybe that's okay. Fine. Here's a question.

Speaker 2 [64:34]

Yeah.

Speaker 1 [64:35]

Yeah.

Speaker 2 [64:36]

A couple of questions. So is there a reason why the generator choose to add a custom field validator for the rejects instead of the built-in pattern? Yeah. In field. Or for the enum, shouldn't that be the field type? Is that a template limitation or an intentional choice?

Speaker 1 [65:04]

Yeah. That's a really good question. I'm not 100% sure actually. I'm sorry. Some of those things that we have that we can do with the pattern, etc. are also not possible. So actually I'm not 100% sure if it's like a limitation of how they implemented the OpenAPI generator, or if it's like, is that a choice? Sorry.

Speaker 2 [65:37]

Do you mark the generated code with some prefix in the directory or file name?

Speaker 1 [65:44]

Sorry, can you repeat the first word? Sorry.

Speaker 2 [65:47]

Yeah. So it's easy to spot. Yeah. Or at least a top level comment. Yeah. Yeah. So the top level comments are automatically there. I can show you. So if you go here, APs, you can see, that's not everywhere, but it's definitely in Maine.

Speaker 1 [66:01]

Here, there's a warning, do not edit this class manually. And how we implemented as Malt is that we have two folders. One is called generated and the other is implementation on the same level. So it's really obvious. And this you can all change when you run the default code. So if you go here, APs, you can see that's not everywhere, but it's definitely in Maine. So what we have to do with Malt is that we have two folders. One is called generated and the other is implementation on the same level. So it's really obvious. And this you can all change when you run the configuration. When you run the open jlnode for the first time, you can add the configuration and you can rename all the folders and you can also create a different folder structure, what maybe I should have done to not change all the time, change directory all the time. Yes. Any other questions?

Speaker 2 [66:51]

Thank you. Yes. So far, just last one for now. Yeah. The frontend and the backend are often in different repositories. Yeah. Where do you put the contract and how do you make it available for both?

Speaker 1 [67:09]

Yeah. Yeah. Yeah. I maybe I lied a bit about that. Yeah. Yeah. So the contract, so we made a rule because the contract is under our, you know, it's like our application, the contract is in our repository. What happens now is that they copy this contract to our, there are the other repository. Yeah. Yeah. Not, not perfect. A lot better than what we had before. That were the questions. Okay. So I have one more slide only. Um, some, some final thoughts. So the, the claim maybe of this whole open API generator is that it's going to be faster and less buggy. Um, so it's not working perfectly, but you can work around it. It's customizing, but you have to learn much test templating. Um, so what I did is that I, this was in December last year, I took five, like, of our main applications. And I looked for those who are still under the design first methodology. Um, so you know, just making the app and coding and don't do anything with the open API generator or to look at the contract first application. Uh, I counted all the releases. Uh, it's anything major, minor patch. And I looked in the change logs, um, to see what kind of changes were made. So most of the changes are of course, like referring to the actual implementation, like changing something to the, I don't know, matching surface or creature surface itself. Um, by far this is not a significant statistical study, but it gives a bit of an idea. So I first counted all the changes that were made to the contract. And in case of design first, that was maybe more, uh, you know, a change to the request model, like directly the Bidentic data models. And then I counted 41 changes for the design first, so the old way, but 24, so that was 18% for contract first. So this gives like a little indication that we can do faster deployments. You know, it's sort of easier to make a little adjustment in the contract, run the generator again, and you make a change and pushing it back. You know, it's like, yeah, there's a little bit more faster. This gives like indication. Then I also counted the bugs related to the contracts. Um, so there were around 24 bug fixes, so related again to the request response model in the design first, but only two for the contract first. And of course, this is different apps by different developers over different timeframes, different maturity level. But I think for us, since we started with one app, we have now five apps that are under the contract first approach, um, gives a bit of support to this feeling that we do think it's faster deployments with less bugs. Um, yeah, so they're already bringing to the end. Sorry, I was a bit faster than expected. I hope you still enjoyed it. Thank you so much. Thank you also, Kat, for supporting me here today with the installation, everything. Um, yeah. Yeah, and maybe if there's some less questions, I'm happy to take them. Thank you. Thank you. Thank you.

Speaker 2 [70:39]

And now it seems like, uh, there are no more questions. Right? Ah. Thanks for the effort.

Speaker 3 [70:58]

So, um, in, in regards to the removal of the field, um, there was the first challenge. And, uh, to make the field optional, we removed the field from the contract. Would it be more, um, you know, uh, interesting, straightforward and less, in my opinion, less anti-partner when we remove the field, we, this field does not exist anymore in the code instead of being optional?

Speaker 1 [71:32]

Um, but maybe someone still wants to send it.

Speaker 3 [71:36]

Yeah. Just considering the deletion, uh, uh, scenario, I want to remove it. Yeah.

Speaker 1 [71:43]

So if, if you, sure, if you want to, if you want to remove it, then, then you completely take it out. But then this might be, because what's required breaks the whole application. So this is something you have to maybe deploy then a new version. Maybe there's first going to be like a second endpoint that has this removed completely. Then all your users can use that endpoint before you then kill like the old one so that you don't like break your whole interface.

Speaker 3 [72:09]

So it makes sense a manual intervention in that case. Um.

Speaker 1 [72:15]

Or the deletion.

Speaker 3 [72:16]

Yeah.

Speaker 1 [72:18]

When you make an incompatible change. Yeah. Yeah. Thank you.

Speaker 4 [72:25]

Thank you.

Speaker 5 [72:38]

Um, you mentioned that on the CI CD pipeline you would compare the two versions. Would you also advocate for doing, for doing it before, um, at the pre hook? Yeah. So either pre commit hook or pre. Yeah.

Speaker 1 [72:55]

Yeah. Then, then, you know directly, but we have it as soon as you push to, uh, like make a merge request. You already see it, but it would be even better because then you notice it even before changing that. Yeah. Sure. Sure. Thank you.

Speaker 5 [73:16]

Thank you. Yeah. Thanks so much for the talk. Mm-hmm. How do you host this endpoint? After this whole, um, creating the endpoint? Yeah. Yeah. Thanks so much for the talk. Um, just a future question. Mm-hmm. How do you host this endpoint after this whole, um, creating the endpoints and how do you host them?

Speaker 4 [73:37]

Do you have any, and most every time people use some things on AWS or some cloud sites, but how do you, based on your experience? Uh, so we host everything, uh, so they're all our application are containerized and then

Speaker 1 [74:05]

with Kubernetes deployed on, uh, GCP. deployed on GCP. But our applications are only open for internal users. So like for our back-end engineering team, there is no, you cannot from the outside use any of our applications. So there is no, I don't know, Cloudflare or something around it. It's only internal users.

Speaker 2 [74:40]

What are the security concerns related to work with OpenAPI in the applications?

Speaker 1 [74:49]

Security concerns? Wow, I have to think about this question. The thing is because it's like in a private cloud. But I, yeah, I, first think about it, I don't think they would be bigger using the OpenAPI generator itself. But maybe this is a too naive answer. Sorry, maybe I'll come back to this later.

Speaker 2 [75:34]

OK. Then we thank you very much. Thank you. Big applause to Erling.