Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
State of Software Quality – API
272 Plays1 year ago
In this episode I chat to Alicia Foreman & Frank Kilcommins, some really great insights!
Get across all the detail of the 2023 State of Software Quality - API report or download the PDF trend report covering the year on year findings.Interested in being one of the first to use a new SmartBear product? Check out SwaggerHub Explore allowing you to quickly view and validate the functionality of your APIs. Learn more and create a free account.
Recommended
Transcript
Introduction and Roles at SmartBear
00:00:00
Speaker
All right, we have Alicia and Frank from SmartBear here today to talk us through something very exciting. But before we do, let's get some introductions. So if you both want to kick off, tell us what you did with that SmartBear.
00:00:17
Speaker
Sure, I'll start, I guess. Yeah, so my name is Frank Kilcommons. I'm an API technical evangelist at SmartPair. So that's basically all things API. So work supporting the product teams, engineering, and then out in the community working with the open API initiative and other folks like that. So I'm a software engineer and architect by trade and yeah, just love API. So I thought no better place to be than SmartPair.
00:00:48
Speaker
My name is Alisha Foreman and Lewis, it's a pleasure to be on your podcast today. I'm also working with Frank at SmartBear in the API lifecycle team. I joined SmartBear just over 12 months ago when SmartBear acquired PacFlow. It's been a really exciting journey that we've been on. I've been working with some of your previous guests, Matt Fellows and Yusef Nabi.
00:01:08
Speaker
They are colleagues and good powers of mind. As you can tell by my accent as well, I'm also located in Melbourne, Australia.
State Software Quality Report Overview
00:01:15
Speaker
I'm looking forward to really talking to you guys today about the State Software Quality Report and all of the insights that we've gathered.
00:01:23
Speaker
Brilliant. Yeah, I wanted to get you on the pod to talk about it because I think obviously there's some great stuff in there related to contract testing. But before we get into that, do you want to give me a kind of brief intro into what the highlights were from the report in general?
00:01:39
Speaker
Absolutely. Maybe if we just start off with a brief little background on the report. SmartBear's been running the State of Software Quality API report now for a number of years and so each year we're putting out this report and we're sharing not only those most recent insights but also looking at trends that we're seeing industry across
00:01:57
Speaker
everything API's from the tools and technologies and methodologies around that through to testing and quality challenges that our audience are seeing in the future as well. This year we had just over a thousand respondents come into our survey, which was fantastic and they sort of range across
00:02:14
Speaker
many industries and lots of different roles as well. Just to give you a bit of a background on those roles, we had quite an increase this year around test automation engineers and QAs, as well as architects and then full stack engineers as well. So that's our representative of that particular audience. Some of the highlights, I think Frank's got a couple to run through, but one that I thought that was really interesting is to just the continued
00:02:38
Speaker
top driver that we see for API growth being microservices. We know that this is a growing area and specifically referencing, I guess, regional or different regions in the world. How they're adopting microservices is really, it's a trend that is growing and changing. We heard this year's report told us that 62% of respondents are attributing their API growth to their microservices.
00:03:06
Speaker
However, about 50% are also believing that there's a top challenge for them to have success around their microservices and that that is in relation to having the right skills and experience amongst their people in their organization.
Challenges in API Quality and Testing
00:03:19
Speaker
I think as organizations are wanting to enhance the digital heart of their business and continue to bring that best experience for their end users, there are demands on the teams that are delivering the solutions that they have to move faster and often with less, especially in our current environment. So ideally we're wanting to make sure that they have the right tools and experience and skills to be able to continue to grow that. Interesting, yeah. Yeah, and maybe to kind of even go
00:03:49
Speaker
into some of the other things that we're seeing kind of coming back being representative across the cohort. The good thing is that there seems to be a positive awareness and outlook with regards to API quality. So API quality does seem to be front and center in everyone's mind.
00:04:06
Speaker
was around 78% of the respondents marked API quality as being either extremely or very important to to the success of their organization and there was good awareness as to the risk of poor quality. So obviously if there's rush design or development and that can lead to poor quality and there's
00:04:27
Speaker
really I suppose an acute awareness as to the organizational impact that can have. Now I'm a firm kind of advocate for the role that people play in in the delivery of API so it was encouraging to see that the impact on the teams and the people within the teams ranked as the number one risk when it comes to poor quality API so I think 49% came back saying that
00:04:50
Speaker
their teams would lose time and spend more time troubleshooting issues in the absence of quality. And there can also be more organizational level impacts as well like losing customers or experiencing brand reputational damage within the market as well.
00:05:08
Speaker
And to address this, everyone seems to be focusing more on API testing. So we did see that the vast majority are implementing kind of rigid testing patterns that are robust and covering the different flavors of APIs. And there's been a 13% decrease in the numbers who have not yet applied an API testing strategy. So that would
00:05:35
Speaker
For me, at least point out that they're already well on their way to having a testing process in place for their APIs to try and maximize the quality.
00:05:44
Speaker
When it comes to the overall kind of challenges that are being faced by the demographic that we surveyed, the top challenges remain the same. Standardization is coming back still as the number one industry challenge that's been faced. And that's not a surprise. It's good to see that security is coming in at number two as well because API attack factors are increasing exponentially. The more APIs we expose, obviously the more surface area that's there to be attacked.
00:06:13
Speaker
And we also saw that composability and multipurpose reuse also scored quite highly in a challenge. And I think this is also pointing back to teams looking to get proof of the return and investment that they have with regards to, let's say, the tactics or the architectures that they've applied over the last number of years to move more into a composable or microservices-based architecture. They now want to see kind of proof on the pudding, so to speak, that that's returning
00:06:42
Speaker
value for their organization as well. One of the interesting things that I kind of took from the overarching survey Lewis is that there seems to be a slight disconnect between the providers and the consumers with regards to what makes a quality API, which I thought was a little bit interesting. So we saw that 78% of the survey group
00:07:03
Speaker
regard to themselves as API providers, so their organization or their team are building APIs, but they're also consuming APIs, and they are very in tune with the needs of an API consumer. But when we looked at, are they able to deliver the attributes that consumers regard as important, which is API availability, API performance, API documentation,
00:07:30
Speaker
And we saw that there was a little bit of a mismatch there so when they put on that consumer hash and critique what they were doing from a provider perspective and only 48% were actually confident that their APIs are well documented and approximately 43% were unsure that their processes would.
00:07:48
Speaker
deliver the developer experience that they themselves would expect of a provider if they were indeed in the consumer seat.
API Protocols and Tooling Needs
00:07:56
Speaker
And I thought that was very, very interesting because it points to kind of the challenge that we have and making life easier for teams to produce high quality APIs. Yeah, definitely. And that falls so nicely into the relationship in contract testing, which I really like and it's really nice that that's coming out of that report as well.
00:08:18
Speaker
And yeah, I think in general, I think a lot of the stuff is what I've been seeing and what I've been talking to people about. Obviously previously we were talking about like open API on the podcast. So yeah, it's good to see those kinds of challenges being faced within the industry and hopefully being tackled by the people that need to be involved. Thank you so much for doing that. So some of the kind of pieces from.
00:08:48
Speaker
that we took out of the report. I think, Frank, you were talking about like multiple protocols and you described it as like a melting pot of protocols being adopted. So what can you elaborate a bit on that and what do you mean by that?
00:09:04
Speaker
Yeah, sure. I think there's quite a lot of kind of content and material out there in the industry. I think it's the nature of software development and testing in general. You know, we all want to move on to the new shiny thing to a certain degree, and then there's a lot of material kind of saying, you know, rest is dead and you need to move on to GraphQL or gRPC or whatever the case may be. But what we're actually seeing in the majority of enterprises is that
00:09:33
Speaker
The modern integration landscape is, as I described, it's a bit of a melting pot of the different protocols and API styles that are out there. There is no silver bullet. I kind of still feel that I can bring this saying in my back pocket coming from a software architecture background and edit it.
00:09:51
Speaker
answer to most questions is it depends and it really does so it depends on the problem that you're trying to solve with the API that you are about to deliver towards certain consumers and so what's the problem that you're trying to solve and what are the constraints of any particular style what are the constraints of the team or the organization that you find yourself in.
00:10:13
Speaker
All of those things kind of combined, plus the consumer expectations with regards to what type of technology and style they're also equipped to be able to deal with, should lead you to the place where you can select the right API style to address the problem. So it's not a one size fits all approach.
00:10:31
Speaker
you need to be aware, and teams need to be aware of the law of instrument. And just because you might be familiar with REST, doesn't mean that REST will be the right API style for every problem that you're trying to solve.
00:10:44
Speaker
And it's never really a comparison versus REST versus SOAP versus GraphQL or whatever the case may be. As I said, they all have pros and cons. So select the right one and don't be afraid to broaden your API stack to be able to cover more of those styles because that can lead you to a place where you have a better overarching experience for your consumers. In certain instances, it'll make perfect sense to have a request response type of interaction.
00:11:10
Speaker
And in others, it will make much more sense to have a more event driven architecture approach. And that's what we're seeing. So what we're seeing is approximately 80% are indicating that they're using one or more protocol or one or more styles and about 60% are using three or more.
00:11:27
Speaker
And that's been backed up, or backed up, should I say, by the popularity of different standards and specifications that accompany some of those different styles. So as you mentioned, OpenAPI is an example there. So that's absolutely the most popular API specification out there right now. And that goes hand in hand with the popularity of REST.
00:11:48
Speaker
We're also seeing JSON schema increase in use. And again, that can also be backed up by the rise that we're seeing in event-driven architecture and the async API specification that's accompanying that in many areas, both open API and async API also leverage JSON schema. So we're seeing kind of a more broad adoption of this within the delivery space as well, which is encouraging.
00:12:15
Speaker
Yeah, I think that's an interesting interaction point with what you're saying about the skills requirements is the fact that like more protocols are being adopted. But one of the biggest challenges is the people and the skills to be able to deliver it. So I think that's quite an interesting insight. So yeah, that's going to be tricky to navigate.
00:12:39
Speaker
Yeah, and I think that's where tooling comes in as well. So I'm a big advocate for API specifications and the affordance that they bring to the provider side of the equation, but also the consumer side. And the beauty about specifications is that they're also machine readable. And so that affords the like, suppose it's a tooling vendor to be able to build
00:13:02
Speaker
tools which can interoperate well with these specifications and hopefully allow a more seamless process for teams who are trying to navigate the whole API landscape.
00:13:16
Speaker
Yeah definitely and brings us on to the first time contract testing being included in the report which I think is amazing and so yeah what kind of trends are we seeing around that now that we have it included.
00:13:32
Speaker
Yes, super excited to have contract testing feature this year. Obviously Parkflow joining the SmartBear family. It's now part of what we do and we've recognised that it's a really important component of testing strategies. So it was included in our questionnaire this year and we had
00:13:52
Speaker
About a fifth of our survey respondents expressed that they're using contract testing. Now this was against a question in relation to the types of testing methodologies that are used around APIs. Interestingly, I guess just as a comparison, and sometimes I guess Lewis, some might explain contract testing as a, you know, form of unit testing.
00:14:11
Speaker
unit testing was really the top point that came out in this particular question that it is the dominant testing technique that we see for a lot of companies. And sometimes I think if you're having a little trouble getting your head around the concept of contract testing, it can actually be useful to sort of think about it in that unit testing style.
00:14:30
Speaker
I guess with that point, you know, understanding that contract testing is there. I'm preaching to the converted, obviously, having people listening to this podcast. Lewis, you can probably rattle all of this off for me, but, you know, it's really going to be able to help organizations restore those service outages. One thing that we did also find out with our survey is that
00:14:51
Speaker
know, for respondents who were asked how quick could they restore their services. We had a percentage of 64 that were able to do that within two hours. Now, if you're adding contract testing as well, that actually increased that number to 66% who were able to restore their services within
Microservices and API Growth Drivers
00:15:10
Speaker
two hours. So we're seeing a really positive correlation there between
00:15:14
Speaker
bringing together contract testing as a means to deliver services or stand up services when they're down a whole lot faster than maybe previously that you would have. And that's obviously paired with a bunch of the other benefits that we see as well. So yeah, really, really positive insight. And I think this year, first year that we've asked about it and then, you know, going forward, we look forward to getting some more trend data around that too.
00:15:40
Speaker
Yeah, I think that's really cool that you can see the actual kind of, yeah, door metrics being observed within, within this report as well, which is really cool. And then, yeah, hopefully we'll see the adoption and kind of the importance of contract nesting increase over time as well, with more people kind of becoming aware of it. I think definitely recently, much more people have been interested in it. And then.
00:16:05
Speaker
with bi-directional, hopefully the adoption will become much easier. And then, yeah, I think we'll see that trend increase over time as well. Absolutely. I think we're all here on a shared mission, aren't we there? Tell the world about contract testing. I'm passionate about APIs and passionate about contract testing. And I do really, it's wonderful to speak to people and hear the evidence of how it can really support it. So looking forward to continuing to report that through this report as well.
00:16:32
Speaker
Yeah definitely. I think it makes sense too like if you look at some of the other kind of challenges and trends that are emerging from the data you know we're seeing evidence that there's a continued shift left approach there's more responsibility falling onto you know API designers and developers and testers everyone wants to be able to
00:16:53
Speaker
release faster microservices is seen as one of the the dominant kind of growth drivers now it doesn't necessarily matter how you slice and dice the microservice if you want to call it a micro monolith it doesn't necessarily matter but what you do want to make sure of is that
00:17:10
Speaker
You can safely extend and enhance that capability over time without necessarily having to suffer the consequences of kind of a major version proliferation. You know, if you don't understand how consumers are interacting with the service area of that API, then all you can assume is that the full surface area manifests itself into the contract.
00:17:32
Speaker
And it can make it a little bit more rigid with regards to how you extend that over time. So the insights that contract testing can provide you very, very early in your iteration, you know, it's excellent to be able to allow you to understand, is this change going to impact anyone? If it does impact anyone, who does it impact?
00:17:54
Speaker
You can have sensible conversations with them, again, back to the people side of it and decide how you want to release that enhancement into the wild. Awesome. So yeah, obviously other things coming out of the report in terms of, and we can not mention it, here's the AI being an emerging driver for API growth. So yeah, just wanted to get your thoughts on that, Frank. How do you see that going in the future?
00:18:20
Speaker
Yeah, it's such a hot topic right now. It's really hard to get away from at least mentioning it, but we did see it creep up. And this year, so in the totality of the report, AI and machine learning are seen as the
00:18:35
Speaker
the number three in the list from a driver perspective across certain industry segments and for smaller company sizes, it was actually up to number two. But I think there's quite a nice networking effect that will be possible and will be happening between APIs and AI. Obviously, we all want to reap the benefits that AI can offer to us and allow us to become more efficient in the work that we're doing.
00:19:04
Speaker
and for it to execute tasks under our instruction with the right level of confidence that it will execute those tasks correctly. And then that's where AI will start interacting with APIs much, much more. And then what we'll have is that we'll have
00:19:23
Speaker
the next wave of creators standing on top of the shoulders of various models and capabilities that will be built and then they'll be exposing APIs on top of that for the next models and so on and so forth. So I see that kind of network effect.
00:19:38
Speaker
happening but API's with regards to making sure that what an AI client can do on behalf of its let's say human peer will be critical because you want to make sure that whatever is executed is validated is critiqued properly and you need to be able to validate and ensure that the results are actually going to be correct
00:20:05
Speaker
and returned back to the human recipient of that information and again that's where the power of APIs can really come into the fold. Nice yeah I think yeah it's a really good way to put it the human side of things is still kind of going to be showing up definitely. Yeah it's like one thing I'm very encouraged by is you know even the let's say the the plugin store or facility built by
00:20:31
Speaker
OpenAI, the core way that the manifest is constructed and pulls in additional capabilities is through an OpenAPI definition, which is also very, very encouraging. So they're putting that front and center. Interested personally because I'm working in it within the workflow specification.
00:20:50
Speaker
that's being matured within the OpenAPI initiative umbrella, which is all about describing workflows across different API endpoints and even multiple OpenAPI source documents. This will even be a nicer way of describing how to execute business value use cases in a machine and human readable way. So I see certain capabilities there
00:21:17
Speaker
to drive kind of API or AI understanding of APIs even further.
Resources and Tools from SmartBear
00:21:22
Speaker
Wow. Yeah. Yeah. That workflow stuff sounds really cool. Sweet. So we'll wrap things up there. I think obviously we've only touched briefly on the aspects of the report. So do you want to tell us where people can read more about it and hear more about it as well?
00:21:40
Speaker
Of course, absolutely happy to. I guess we've got two great resources that anyone who wants to dive into the findings that we've shared today a little further, got a microsite and you'll find that at smartbear.com. It'll be linked from the homepage or if you just search State of Software Quality API SmartBear, that will pop up. That'll give you the findings from this most recent report and then
00:22:03
Speaker
There's also links on that page where you can download the full PDF report. And that has got year-on-year trend data, which can be quite insightful around some of those aspects of the things that we've spoken about today. Brilliant stuff. Yeah, there's information there. So I really would encourage anyone to get access to that data. It's openly available. Some really nice insights that of course is way broader than we can cover in our very quick conversation.
00:22:29
Speaker
I know I was lost in the data for a week or two anyway. I was fascinated about different industry SKUs and trying to see some trends. So yeah, great resource to have. Really cool. Yeah. Yeah. I think some of the stuff you pulled out is really, really cool. Great. Anything else that you guys wanted to promote?
00:22:48
Speaker
Well, I can. We have a brand new tool out. Well, brand new. It's eight weeks old at this point. It's called Swagger Hub Explorer, but it's a free tool to explore APIs. And I guess I was chatting with Alicia before and looking at, we were talking about some of the findings that the report highlights as to what their expectations are with regards to API tooling.
00:23:15
Speaker
And one of the things that they want to be able to do is to be able to explore APIs easily. They want to be able to ensure that there's integrations with some of their existing tools. And this is one of the ways we leverage these reports is that we try and really uncover what's important, what folks are challenged with, and take that into our own learnings and make sure that that feeds into the next wave of products and capabilities that we're doing.
00:23:44
Speaker
That's a nice little free tool out there if folks want to try it out and send us some feedback. We're quite young on our journey there so people can really jump on board and help steer the direction of that product. Brilliant. Well, thank you so much both for coming and joining me on the podcast. Yeah, I really appreciate it. Thanks so much, Lewis. Real pleasure. Thank you. Yeah, thanks for having us, Lewis. Lovely to chat to you.