Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Meet Matt Fellows - Co-founder at Pactflow
160 Plays2 years ago
In this episode we have Matt Fellows co-founder of Pactflow, we discuss starting Pactflow to recently being acquired by SmartBear, the pact open source community and maintaining multiple languages. We also touch on personas in Contract Testing, performance budget in your pipelines. Find out more about Pactflow or sign up for free. Also checkout the Tutorials, CI/CD (Consumer Driven Contract Testing) workshop, Bi-Directional Contract Testing Workshop and Pactflow examples.
Hosted on Acast. See acast.com/privacy for more information.
Recommended
Transcript
Introduction to Matt Fellows and PacFlow
00:00:01
Speaker
Hello and welcome to this episode of the podcast. We have Matt Fellows, co-founder of PacFlow. We discuss starting PacFlow to recently being acquired by SmartBear. We discuss the open source community and maintaining multiple languages. We also touch on the personas within contract testing and integrating a performance budget within your pipeline. It's full of great conversation and I'm so happy to get Matt on the podcast. It's a really good one, so enjoy.
Matt's Career Journey
00:00:31
Speaker
Hello, Matt. Welcome to the podcast. Yeah. Good day, Lewis. Thanks for having me. It's, uh, it's great to be here. My first podcast. You're first. That's what surprises me. I think you've got a few firsts now, but yeah, I think I've done lots of public speaking, but I've never done a, uh, to my knowledge, I've never done a podcast. Cool. So yeah, tell us a little bit about yourself. What did you do before you got into PACT and contract testing?
00:00:57
Speaker
Yeah, I guess I've been in the industry about, about 20 years, I think now almost. Yeah. My last job, I was a principal consultant at a, at a company called Diaz. Basically we're an Australian or we're an Australian boutique product development, you know, consultancy kind of did end to end.
00:01:13
Speaker
development from user experience, design, obviously through all the engineering cloud and then out to market as well and even actually build some physical products. So that's a really cool place to work. I love working there. I worked there for about seven and a half years and I got to work at some of Australia's largest companies like Seek Jobs, National Australia
Transition to Cloud Applications with PACT
00:01:32
Speaker
Bank.
00:01:32
Speaker
the NIB, large insurance company, the University of Melbourne in Sweepero near the end of my tenure there working on some IT projects. So I kind of worked to the full spectrum of cloud and transition to cloud and DevOps and obviously APIs and microservices. And it was actually kind of on my first client at DS, at its seat, was where I first got exposed to PACT, actually, and on my project. And so we were taking quite a large monolithic .NET application, actually, of all things.
00:01:59
Speaker
And, you know, we're splitting that and deploying that into the cloud. And that was their first public cloud workload, you know, their first real foray into kind of DevOps and all of this kind of agile delivery as well as microservices. So kind of had the big three going on there. And, you know, contract testing was this thing we're looking at doing with this new tool called PACT, which I'd never heard of at this point in time.
00:02:19
Speaker
You know, at the end of that project, it was really formative for me in my career, I think going forward because just in, it was only a couple of months project, the first one anyway. And, um, you know, they went from deploying once every couple of months, this big thing. It was a huge, you know, it was a huge effort of multi-teams. You get to get a big whiteboard, right? It's sort of the train view of the world.
00:02:38
Speaker
all the different scrum teams would come together and try to coordinate what they would need to do to get this release out the door. And, you know, inevitably when the release went out, you know, take hours to do the actual release, something would go wrong somewhere and it would all need to be backed back out. Right. And you'd have to sort of figure it out. So going from that to.
00:02:53
Speaker
literally deploying in seven minutes. So commit hit the commit seven minutes later, it was in front of a customer. It was just transformative, uh, you know, for that particular team. And, you know, I was just super energized by seeing how that all worked together. And, you know, packed was part of it. It wasn't that the main thing, there was some DevOps stuff there, obviously in terms of ways of working in the cloud, but the contract testing was really key to, to shift in the way we actually built and test software and critical to obviously
PacFlow's Acquisition by SmartBear
00:03:17
Speaker
the release pipeline. So, you know, I got really excited by that.
00:03:20
Speaker
I think the MD of Australia, we showed him through that one day. We actually got him to do a commit. He didn't really know what he was doing, but then like we kind of took him through the wall and we showed him the build lights and things we had going. And you know, we sort of watched some flash from orange while it was building and then went back to green and we're like, oh, come look at this. And we showed him the website. He just changed and he just couldn't understand what was going on. Like he's like, this is a real website. You know, what have I done? And then he's like, you know, every team needs to do this. And it was kind of just a really good example of
00:03:47
Speaker
There's lots of great learnings there, but it was just a good example of showing what good looks like. And, you know, sometimes teams will go, that's what we want, right? Because I guess the foray for me into PACT, at that point in time, PACT.js, which is one of the languages I helped maintain was really rudimentary. And so I thought, I'll put a couple of contributions in there to help with it. And, you know, seven years later, I'm still a maintainer of all various parts of PACT.
00:04:10
Speaker
And it was quite funny because you know, Melbourne's on a very big development scene really in the grand scheme of things. And yeah. So Sikh was just around the corner from realestate.com.au. That was where it was born, just around the corner. So it kind of all full circle. Yeah. So that's the start of your journey. Let's jump in to here and now and exciting news that you've just been acquired by SmartBear. So yeah, tell us a little bit about how that came about and what that means for, for PacFlow.
00:04:39
Speaker
Yeah, it's great news. It's very exciting. We're still kind of in the early stages where we're very much in the, you know, that kind of romance stage. We've gone through the acquisition part of it and we're now in the, we've just landed at, you know, only a couple of weeks ago, really officially as SmartBear employees.
Values and Community Alignment
00:04:56
Speaker
Yeah, so far, everyone's been fantastic. Everyone's been really friendly, but kind of how we got there was, you know, despite pack for itself only being a commercial offering for, you know, three years by the time we sort of, I guess exited, you know, we've been working in contract testing since, you know, all of us since 2013. So it's, it's been quite a long journey, you know, we never got into this.
00:05:16
Speaker
you know, to sell a business that wasn't why we, you know, we did this in the first place. But, you know, I guess if you think about the product itself has matured, has been in the market for a while and the concept has matured. And I think a few things came together that sort of meant that was came to fruition. One was that, you know, obviously we, you know, we obviously look at our competitors, we look at other tools in the market and
00:05:35
Speaker
Smart Bears, one of the big players there has a lot of great heritage in the testing space and a lot of great tools in that space as well. So that's one angle. We're almost releasing the bi-directional contract testing feature. So that was that sort of more of a, I guess a play to help people do what we'd call design first or provider driven.
00:05:55
Speaker
kind of API design. And that feature works really closely, or the initial version of it works really closely with an open API spec, Swagger for those that term. And we're looking at integrations there. And of course, Swagger Hub, we've spoken to a lot of customers who are using Swagger Hub and you would raise that into a customer call. So it was just one of those things that we thought it would reach out. And I think in the end, I'm not sure which direction the reach out came from was from SmartBear or from us. But there'd been a bit of kind of playing through different channels.
00:06:23
Speaker
Yeah I guess conversation came together that was the first chat and then you know through a few conversations we just realized there was a lot of alignment in values there and SmartBear also had you know they've got a very large global team, they're getting feedback from their customers all the time and contract testing just comes up a lot.
00:06:39
Speaker
And turned out that another team internally was actually looking to use contract testing on their own internal services in terms of delivering software. You know, just from a few different angles, you know, it just kept coming up and they, they basically indicated that, you know, they're going to go down this path at some point. Contract testing is definitely a thing.
00:06:55
Speaker
And, you know, they really want to make that fit into their ecosystem somehow. You know, through our conversations, we were just, you know, I guess we're pretty impressed with their open source heritage. Yeah. So again, like open API spec, the Cucumber community, the SOPUI community that is a really active, you know, well-funded communities and, you know, funded by SmartBear, probably first and foremost on how, on our things we cared about was the open source community. So we felt like there was good alignment there. They want to push in recognition of some of the ways that testing is changing and their tools maybe it needs to shift.
00:07:23
Speaker
with that and the tools are shifting with that, you know, that focus on developer-led testing. And of course, just that general, you know, general values around prioritizing software quality, recognizing that testers and developers have a role to play across that whole spectrum. And it's really just quality across the entire lifecycle of software meant that if we could, if we can make this work, you know, contract testing is a narrow thing, but if you can make it work within a bigger picture, then our customers are going to
Scaling Open Source and Maintaining PACT
00:07:48
Speaker
do better. So,
00:07:49
Speaker
those kinds of things all came together and you know obviously we had to work through new commercials and things like this but you know we all felt like the open source community would do better the product would do better and our people would do better and so you know ultimately it just made sense to us cool yeah that's great to hear about the alignment that you guys have and what that means for the future because i think it's always a it's always a scary moment when a tool that you love gets taken over by a vigor player but yeah it's great to
00:08:15
Speaker
Oh, absolutely. Yeah. I mean, it's one of those things. It's such a big decision. And we, we, we generally feel the weight of the community with these things. Like even sometimes just putting a post in Slack, we're like, well, is this going to be the, is this the right thing for our community? Is this going to benefit the community or, or not? And so something as big as this was, you know, there's a lot of conversations, a lot of conversations.
00:08:36
Speaker
You know, we had to we had to get we had to get to the core of all of these things. And I have to say that the the people we're working with in the smart best side were really professional and really supportive and really open about everything. And hopefully we reciprocated there and we really felt like it was a good thing to do.
00:08:51
Speaker
And you touched on the community that are involved with obviously maintaining the open source side of Pat. What are the kind of challenges that you face in scaling that? Obviously you started, when you started, there's a couple of people using it, company obviously using it in-house. So yeah, how did you kind of scale that operation?
00:09:12
Speaker
Well, I still think we're doing that. I do believe that I think process is important. And I think historically we've been fairly poor with processes as such a small group. But, you know, I think we can continue to do better there, but I think process helps because you can be really clear to everybody what the process is. And again, I don't think we're necessarily fully realizing that those
00:09:33
Speaker
those ideals, but you know, if you can say, well, you know, when a bug gets raised, this is a process to deal with it or a feature request will raise. It's a process to deal with it. That just helps because there is a large community of non-commercial members, if you will, who want to contribute to the software. And they do that for all kinds of reasons. You know, we got into it because we loved it. It was really interesting. Well, I did. We all got into it for slightly different reasons.
00:09:54
Speaker
you know some people just like it because they get to contribute to something that people use and some some are happy to do that but they're really focused on doing really interesting engineering work and maybe less focused on shipping features quickly and so there's all these different dynamics that come to play and I think transparency and process help you navigate through that a little bit.
00:10:11
Speaker
But to be honest, that was one of the reasons why we looked at someone like SmartBear as well, because they do have quite large communities and they have processes to deal with that. But in terms of how we're looking to deal with that, I think process is definitely one of them.
Enhancing Documentation Strategies
00:10:22
Speaker
The second one is looking at our product surface area. So PACT is not quite unique, but it's fairly unique in the sense that we have to build a language binding for each one. And the complexity in that language binding is not something to sneeze at. It's quite complex.
00:10:36
Speaker
Even though we've got, you know, architectural things underway, you know, the rust core, we've got that we would distribute across it and plugins and things like this. There's still a reasonable amount of work just to keep each client library up to date. And of course, there's a challenge in getting those, those up, those all kind of a feature parody. So that's one of our big, big few priorities this year. But, you know, reducing the surface area and carefully controlling that is something for us to think about because
00:10:59
Speaker
Basically the more things you build with documentation or features or you know tools or whatever it's more things to maintain and more things that very quickly suffer entropy and just all of a sudden they drifted from the versions of node or out of date or something or the library that they depend on and I've all got these vulnerabilities in them so.
00:11:15
Speaker
Yeah. There's a lot of things in that. So, you know, automation is obviously going to be key for us as well. And probably the third area is just having, having at least one person dedicated to shepherding and looking after the community. And so I know you spoke to Yousef, Yousef Nabi on a previous podcast, but you know, we just felt like he was a great, a great fit for the team for all kinds of reasons. We can talk about that, but having somebody who, I guess rewarded, if you will, you know, commercially rewarded in the sense that it's your job is to make sure this community is looked after.
00:11:43
Speaker
advocate for that community, make sure that we're focused on the most important things, solving the most important problems. That's actually just a big thing in and of itself. And the strategic things that we need to work on to, you know, documentation is going to be really important for us. And I think we're probably up to our fifth large documentation rewrite that we'll need to be doing very soon, particularly given the commercial side of the business and how that navigates and interacts with
00:12:05
Speaker
the open source side. Those two things I think are really, really important and documentation. We're up to the point, I think where we are in terms of the world is that documentation really is a feature. It's not kind of this side thing. It's like actually it's part of the product and the tools that do documentation really well. Developers are going to navigate and engineers are going to navigate to documentation and tools that are
User Journey and Framework Integration Challenges
00:12:26
Speaker
really easy to use. This is all part of caption that sort of developer experience. Focusing on that we believe will actually lead to better usage and better outcomes for everybody.
00:12:34
Speaker
coming from obviously contributing to the packed source. For me, it was really easy because it's well tested. So it was very easy for me to come in and look at the test and understand what was going on. And then for me, the documentation was actually really clear about how to create a pull request and then get the feedback that you needed by pestering you or someone else in the community to review. So for me, that was a really enjoyable experience.
00:13:01
Speaker
I guess one of the challenges that I see with it is that people always ask me about the end-to-end journey. I guess that's where the
00:13:10
Speaker
the different frameworks, it's quite difficult to understand, okay, once I've written the tests over here, what do I do next? Do you know what I mean? And so I guess that's one of the challenges with the documentation side of things is because you're in a GitHub repo, creating your consumer test, and then you publish it and you're like, oh, now I need to go to a different repo to understand what I need to do. Whereas people like it all in one place, don't they?
00:13:36
Speaker
Yeah, exactly. And I think that comes back to the surface area and being able to control the narrative a little bit. Historically, each project was sort of, you know, managed individually, separately by the maintainers and the documentation was written by those maintainers differently. So there's no, there's no consistency or there wasn't any consistency in each of those. And that's, that's slowly being worked on. But I do think that what we would like is a, is a, is a one guide to rule them all. And essentially you can go, I want the Python version of that or the .NET version of that or the Golang version, so on.
00:14:04
Speaker
That would help and then you can also navigate between them a bit better, the concepts are clearer. And then you're also right, then moving from a single language or moving from the client language to the broker. Really, there's a big difference there as well in terms of the personas who get involved with the product. So in smaller companies, it's often the lead developer or lead tester or Estet or someone.
00:14:24
Speaker
who will do that work. In a bigger company, we actually find there's huge distances between the people writing the tests and those actually doing the work to integrate it. You might have a separate team who run the packed broker, but have absolutely no idea how it gets used. You have a separate team in between who then set up all the pipelines. And then of course you've got the engineers and the testers on the ground who write the test and maintain packed. So you almost have three separate teams or personas who get involved.
00:14:49
Speaker
And the documentation is very much not written for the first 2% as I just talked about. It's really targeted at the engineering level. But if you're a, you know, let's say you're a DevOps person, hate the term, but if you're a DevOps person and you're on those DevOps teams who have to do that, then, and maybe you've come from a different background where engineering, code engineering is not your thing and infrastructure as code is probably as far as you go.
Contract Testing in Deployment Pipelines
00:15:11
Speaker
Understanding those contract testing concepts and what that means is actually a bit of a leap as well. So we do have those different personas we need to sort of target differently and make sure that we've got the right coverage. But equally, we need the journey to be quite seamless. So we've got the work cut out for us. And I think that's just a challenge that maybe it's just interesting where we sit in the landscape of tools, just like anybody. There's no real tool that fits in a single box usually.
00:15:36
Speaker
So we kind of get lumped in the API testing box. And that's definitely true. We sort of fit there, but it's very blurry when that moves into the CI CD box, because we're really critical to helping you give that information around what's, what's deployable. And again, that, I think long-term, you can see how that would work really well with something like smart Baroness tools and ecosystems where the can I deploy feature right now is purely focused on the contract. But of course that's not the whole picture, right? If you're deploying software,
00:16:02
Speaker
you're probably not just relying on that one single thing. And the pack, you would have other tests and other checks and balances in place. And you're kind of deploy is one of those. But you can imagine a world where that's, that's an a richer, it's got richer information associated with that. Yeah. So, you know, I think conveying that story and conveying how to do that, we've constantly been iterating on and we're not making everybody happy. But you know, that's, that's, it's an ongoing project.
00:16:27
Speaker
Yeah, no, that's a really cool idea to expand out the Can I Deploy feature to other areas of your software. I think that's a really great idea. In terms of where contract testing sits, I always talk about the benefits it has in terms of communication. And it's like, it's basically like an agile tool. Obviously you've got the framework, but the framework isn't doing most of the work. The people around it and the communication, the interactions you're having between teams is what's doing the work.
00:16:57
Speaker
and making your software better over time. So I think that's one of the great things about it is that it's not necessarily the code that's doing the hard work. It's the actually building the quality around it.
00:17:11
Speaker
Yeah, a hundred percent. I think just going back to literally my first, my first touch point with PacDeva back in 2014 on that project. We had a, we had a QA on the team. His name was Joel. And, you know, actually when he started doing this, we, you know, one of the first things we did was we looked at, you know, let's say we looked at the pyramid. We know it's not the best heuristic, but it's people understand it.
00:17:32
Speaker
And we said, okay, what tests do we currently have? And what we found was there was a huge set of unit tests that the, the, the dev group in that same squad wrote. And there was a separate entirely, you know, quite a large integration testing pipeline. And what I learned through several years of consulting after that was that.
00:17:48
Speaker
The state of the industry generally meant that, who did this? The state of the industry was that most of the time, those integration pipelines in whatever form they were called, were pretty much red, you know, 90
Performance Budgets and Recovery Optimization
00:17:59
Speaker
% of the time. And they kind of magically went green just before release happened, right? Because, and somebody, usually the tester, the poor tester had to go through and just make sure it all worked. They'd have to go in there and make sure the data was there, make sure the test cases were still valid, that the functionality was tested.
00:18:15
Speaker
all that stuff. And they were still, even though they're inside the same team, they were still operating separately. And, you know, there's some benefits to those checks and balances in that way. But what we did was we looked at that and said, well, there's a huge amount of overlap there. That's kind of wasteful. We don't need that overlap. And we basically said, okay, these ones here, here's all the tests that are running really slowly.
00:18:33
Speaker
And it costing us time in our pipeline. We actually put a performance budget into our pipeline. And, you know, about 15 minutes is usually what I would set a budget out in my CI pipelines or my CD pipelines, because anything longer than 15 minutes now, when you've got a production issue, it's minimum that time plus, you know, how long it takes to fix a bug. So your mean time to recovery starts to fall apart. And since then we've learned the mean time to recovery is actually a really good thing to optimize for.
00:18:56
Speaker
So anyway, we did that and we started to cut out all the tests that we felt weren't really serving us. And we erred on the side of, we'd rather get decent coverage and feel pretty comfortable and use contract testing to give us some extra, you know, confidences. Fix things really quickly if they come up, knowing that they're probably not going to be big issues given that we're testing them and we're getting these out really quickly and we're constantly getting in front of product people.
00:19:19
Speaker
rather than try to cover everything beforehand. And of course the context here is we're talking about a web application that was, this is kind of a similar to a LinkedIn type product. So the consequences of a small bug weren't going to be financially or commercially terrible. It wasn't like a payment would go to the wrong bank account or something and all of a sudden
00:19:38
Speaker
someone has lost a million dollars. So that context helps, I think. But we cut it all back. And, you know, of course, coming back to this whole thing around the roles, Joel's thinking of feeling a bit uncomfortable here that we've kind of just removed most of what he was doing as a job, right? What is my job now? But, you know, fast forward a little bit. At the end of it all, he realized, hang on, all those automation tests, actually a lot of that work was not very interesting.
00:19:59
Speaker
for starters, and he was just spent time debugging code and tests. And what he got to do at the end of it was actually focus on the value and focus on coaching and focus on much higher level value. So what he was doing was valuable, but not very valuable. If that made sense for us, he actually got to use his brain. I think this is what he said was I got to use my brain. Oh, I now I get to use my brain, you know.
00:20:19
Speaker
And you know, that was, he went from this sort of not very comfortable about what was going on here.
Joel's Journey from Testing to Coaching
00:20:24
Speaker
And I wouldn't say he was detracting from the process. He definitely wasn't that far, but yeah, he was obviously concerned about his role. At the end of it, wasn't concerned about his role at all. And really what happened was we just got a better product at the end of it because he got to spend, use his brain to solve more complex and more difficult problems that an automation test was never going to pick up in the first place.
00:20:43
Speaker
So I just think it was a really good story. And I think, you know, that that's kind of what taking contract testing aside, this is what you want to do with testing, right? You want to get to the point where you're constantly working on higher value tests and not spending too much time on low value tests where you can avoid it.
Community Involvement Beyond Code Contributions
00:20:56
Speaker
Cool. That's a really great story. Let's wrap this up. How can the community get involved in PACT and
00:21:02
Speaker
We're always wanting people to get involved as there's heaps of different ways you can get involved. I do think a lot of people think that the only way you can contribute to open source is to contribute code. But as you probably heard just now, you know, actually documentation and helping people understand how to use a tool is in many ways more valuable than writing code because
00:21:20
Speaker
Code you have not discovered the features you've not discovered yet and you've moved away from a tool because they weren't clear then no one's getting the value from it anyway so a good example that i think with practice is our messaging support we don't do a very good job of communicating that but we actually support it if you want to have the keys or what have you can do it but it's most people really don't know that.
00:21:41
Speaker
So I think there's a lot of places you can get involved. What I would say, if you do, come and join our Slack channel at sack.packed.io. Wave your hand there and say, this is a great project. I'm really keen to get involved. But here's what I, you know, what I'm comfortable doing, whether that's working in Python or .NET or Go, or if it's writing documentation, or if it's just looking for bugs, there's always, there's always things to do. Wave your hand myself, use it for one of the other maintainers. We'll, we'll reach out and we'll, we'll, we'll connect and we'll find something to, to support you with.
00:22:07
Speaker
Cool.
Learning Resources and Future Plans
00:22:08
Speaker
Yeah, that's great. And I think adding to the documentation, the readme file is a first great commit because you get to understand the process and then you can build on that. Cool. And where can people learn more about PacFlow?
00:22:21
Speaker
So from a PackFlow point of view, you can visit packedflow.io. That's our commercial website that talks about all things to do with our commercial product. From there, there's heaps of links to our documentation and whatnot. If you want to learn about contract testing, we have a bunch of courses and material on docs.packflow.io. That'll often link back to the open source documentation as well, but we've got a bunch of videos and workshops and tutorials you can do, browser and offline, and that's a really great place to get going.
00:22:49
Speaker
Of course, once you've done that, come and join our Slack channel and you have a chat with the community there. We've got about three and a half thousand members as well. So yeah, come and say hi and we'd love to welcome you to our community.
00:23:01
Speaker
Yeah, I think the select channel is a great place to come with your specific examples and we can apply that to your context. There's some great people who have been through it and you'll gain insight from them. And yeah, if you like hearing Matt's voice and like his approach, then yeah, the videos are a great way to see more of Matt. I'm very much hoping you so far somebody else can record those videos. Thank you so much for coming on the podcast and yeah, really appreciate it.
00:23:29
Speaker
You're welcome. Thanks, Lewis. Yeah, it's been, it's been great fun. We'll talk to you soon. Thank you. So this is going to be the last episode of the current series of the how to get started with contract testing podcast. I'll be taking a short break, then we'll be coming back with new episodes. I've already had a couple of requests for topics, but if you have anything you would like to know about contract testing or API testing in general,
00:23:53
Speaker
Then reach out to me again via my website at pacman.co.uk and I'll get an expert guest to discuss that topic. Thank you very much. Thanks for listening to the podcast. Please subscribe and leave a review. Share the podcast with your colleagues and friends. Visit my website at pacman.co.uk. Also where you will find my blog and courses. And you can now support me also on Patreon.
00:24:20
Speaker
after consultancy services will happily share my experiences with contract testing in your company. So contact me via the website and we can talk.