Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Bridging the Gap Between Languages (with Martin Johansen)
1.7k Plays10 months ago
If you ever feel overwhelmed by the number of different programming languages, this week’s episode might just offer you some solace, as we talk about an attempt to reunify many of the most popular languages by focussing on the bread & butter things that every language supports.
I’m joined by Martin Johansen, who’s been working on a new tool called Progsbase. With it, he’s created a spec based on all the things programming languages can agree on, and is building a library that can cross-compile between them. Write a program in Java, and it can be automatically translated to PHP, Python and a great deal more.
But how far can he take that idea? Is there really enough unity between these languages to build something universal? How do you bridge the divide between manual memory management languages like C and garbage-collected ones like Java? And what would it actually feel like to write code this way? Let’s put Martin’s plan under the spotlight and find out…
–
Martin on Twitter: https://twitter.com/martinfj
Kris on Twitter: https://twitter.com/krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Progsbase homepage: https://www.progsbase.com/
The Spec: https://www.progsbase.com/docs/programs/
The Progsbase library repository: https://repo.progsbase.com/
The Bug Bounty: https://www.progsbase.com/bug-bounty/
–
#software #programming #podcast #programminglanguages
Recommended
Transcript
Are all programming languages essentially the same?
00:00:00
Speaker
There's an argument people make in programming all the time, that all programming languages are basically the same. You add a feature here, you make a syntax change there, but essentially all programming languages are the same thing. Now I have to say I disagree, I think the differences are where things start to get really interesting. But you have to concede, there's a lot of overlap.
00:00:22
Speaker
Java and C sharp have obvious similarities because of an obvious shared history. If you know Perl and I teach you Python, I'd hope you'd like Python, but I wouldn't expect it to completely blow your mind. In the great Venn diagram of programming languages, there's overlap, of course there
Martin Johansson and cross-language code translation
00:00:41
Speaker
is. And the question this week is how much overlap and how far can you push it?
00:00:48
Speaker
I'm joined this week by Martin Johansson and he's created a language slash tool that's trying to exploit the overlap between languages and build a kind of bridge so you could write something in Java and translate it into a similar language like C or a less similar language like PHP. He's got 13 languages he can bridge between so far and
00:01:13
Speaker
It's a curious idea I think it needs digging into, so let's get going, hear what he's got to say. I'm your host, Chris Jenkins. This is Developer Voices, and today's voice is Martin Johansson.
00:01:38
Speaker
I'm joined today by Martin Johansson. Martin, how are things? Things are fine, thank you. Thanks for having me on. Pleasure. Pleasure, while you're out there in the snow in Oslo, I believe. Yeah, it's not really a snowy hill outside right now, and it's quite beautiful. It's a beautiful place, Oslo, especially in the winter. Although it's perishingly cold, I'm not sure I could cope with living there. Yeah, it's at the end of a cold spell right now. It was 20 degrees Celsius for a few weeks, so that's very cold.
What is ProgSpace and its purpose?
00:02:05
Speaker
Minus 20.
00:02:07
Speaker
Yeah, exactly, minus 20 cents. I don't normally think that side of the zero. So the reason I've got you on is ambition, I think. You've been trying to solve a problem that I think everyone's tried to solve, but the way you're doing it really interests me. So let me put it this way. Everybody, so many companies, so many people have said that they can solve the reusable software problem.
00:02:33
Speaker
And that's what you're kind of going at, but from a new angle, an angle I've not heard about before. Yeah, exactly. So this problem has interested me for a long time. And what started me out was that we know that we write these text files with code most often, and those don't change. But what we see is that either the programs will stop working over time because we need the software updates, the OS updates, or a new language is trending and people are moving over to that language, which require us to rewrite the code.
00:03:04
Speaker
And this was very interesting to me. I started a series project to investigate why this is. And that's where I came up with the technology called ProgSpace and the theory behind it. So I just dig into what it is and what my thinking process was. Give me your general approach and then we can dive into the details. Yeah, so it is a programming language called ProgSpace and there are related tooling that we can talk about later.
00:03:30
Speaker
The approach I took when creating this language was very different from what others do. Instead of trying to build a new language with strategies and mechanisms that are new and that I thought was better than other languages, I took in a sense the opposite approach. I looked at languages and saw what do we actually agree on.
ProgSpace's core features and language constructs
00:03:54
Speaker
And it started out as an experiment. Let's try to build a language with those things that most people and most developers agree is a good idea. And we also see that when a language designer creates a new language, he will include these mechanisms in the language.
00:04:12
Speaker
For some reason. What kind of things are we talking about? We are talking about very basic stuff here, like arrays, functions, structures, a character data type, a number data type, a Boolean data type.
00:04:28
Speaker
Okay, so we can agree on primitives before it all breaks down into flamewars. Exactly. Yeah, exactly. So what started out as an experiment where I picked up those mechanisms, those language constructs that people agree on, I built it into a language and I started coding it. And what I found out is that it was very strange to code in this language in the beginning. And I've heard that from other people who have developed it as well, that it is a bit weird in the beginning.
00:04:57
Speaker
But then once you get into the flow, something interesting happens. You will see that you understand the language 100% now. And you can just focus on the problem you're solving. But that sounds like you've solved the problem by just creating another language. Yeah, so one interesting thing about this language is that it does not have its own syntax or its own compiler. What you actually do instead is that you write code in, for example, Java.
00:05:27
Speaker
But you write in a subset of the language using only certain language constructs. So what you write is just plain Java. You compile it with Java, static analysis with Java, everything is Java. But then you can pass it through an announcer, thus I have made a proxy space announcer analyzer, and it will tell you whether it is valid proxy space or not.
00:05:51
Speaker
Right, so ProgSpace isn't its own language, it's a subset of Java. It's a subset of many languages, in fact. At this point in time, I have 13 languages that I have support for. 13. Yeah, and actually, you could choose any of these languages to write ProgSpace in. At the current time, I only have Java as an input language, but as an output language, I have all the 13 languages.
00:06:19
Speaker
So you're saying I can write a subset of Java, and if it's a valid subset of Java, you can take that in and spit it back out as 13 languages. Exactly, exactly. And this includes also the test cases, right? So what is interesting here is that not only do you code the code itself, you code the tests as well. So not only can you spit out in a new language, you can also automatically run the tests.
00:06:46
Speaker
and see that the tests pass in all the languages. So it's actually quite interesting to see this in action where you will have a list of languages and it will just pass, pass, pass, pass going downwards. And you just made a change to Java, the Java version of it.
00:07:00
Speaker
Okay, let's hammer on this idea. So the first, if you're going from, give me two or three of the languages. So Java, did you say Python is in there? Yep, C++, Python, Visual Basic. I can bring up the whole list if you want. No, no, it's okay. It's just enough to get me started on the question. JavaScript, of course, is an important one, TypeScript. JavaScript and TypeScript.
00:07:26
Speaker
If you've got, how small a subset of the language are we talking? How much common ground between those languages that you could usefully translate? Yeah, so it's important to note that I did not choose the languages first and then selected the subsets because there are languages outside that I can translate to, for example Haskell. I cannot translate to Haskell. But I can translate to those languages I mentioned.
00:07:56
Speaker
What was your question answer? I'm just I'm wondering what okay, let's put it this way
00:08:02
Speaker
You're saying there's enough common ground between Java and Python and C++ that you can translate automatically between them, but not Haskell. So what's that boundary line where the common ground disappears? Right. So I don't know exactly where the common ground disappears because what I did was to select those language constructs that seemed to be low in disagreement among developers.
00:08:28
Speaker
Right. And then, and then I selected these constructs based on what you need when you program as well. Right. Okay. So therefore I ended up with this set of constructs that are very practical. It's very easy to work with them, but they're, they're also common among 13. There's actually a lot more, but I've only worked with the top 13 languages. Okay. When I did this. Can you give me an example of something you thought was common and sort of failed to pass that test?
00:08:59
Speaker
Good question. Yeah. One big one is the integer data type. Oh, really? That is actually a very interesting one because there is actually a lot of corner cases when it comes to integer math.
00:09:15
Speaker
that makes it difficult. And this was one of the more difficult parts when I started making the language. But once I got into it and saw that we can use floating point numbers for most things, and we actually do that when you write JavaScript, there it's uncontroversial. It's probably controversial, but it's less
00:09:36
Speaker
less controversial. It's not like you have a choice.
Handling data and memory in ProgSpace
00:09:38
Speaker
Yeah, exactly. But what happens is that when you get into it and start coding, you will see that you always know what will happen when you divide two numbers. For example, in Python, if you divide by a negative number, it behaves differently than if you divide by a negative number in Java. So that's an example where they disagree. Right. Doesn't that mean then you can't have division by negative numbers in your language?
00:10:03
Speaker
But when you do floating point numbers, it's not a problem, because then people agree what should happen. Okay. So then we've got to get into what the practical size of this subset is. I mean, if I'm writing, if hypothetically I took JavaScript as my source language,
00:10:23
Speaker
that limits the specificity I can have in my target language of Java. That's right. So now we get into an interesting thing because what I saw early on was that we can roughly divide our programming tasks into two categories.
00:10:40
Speaker
One category I called computational, and the other, there was no term for it, so I meant the term called infrastructural, but it basically means where you interact with hardware. For example, the screen, a disk, a keyboard, a clock device. Whenever you interact with any of these devices, I separated that out into separate fields.
00:11:01
Speaker
And then I focused only what you can do with memory and your CPU, running on a single thread, because I also regard multithreading as interacting with different devices.
00:11:15
Speaker
Okay, so is this similar to the division between pure functions and side effects? Yes, it's very related because when I started working on this, what you can do with just the CPU and memory, what you write is actually pure functions. In my language, you're allowed to change the input parameters. So I guess that doesn't fit the strict purity requirement. But it is the case that the function will always return the same output for the same input.
00:11:45
Speaker
OK, yeah. And there is no global state either. So it's always the case. So the functions are pure in the sense that it will always return the same output for the same inputs. OK, so pure functions. Let me think of something that falls awkwardly outside of that. See if I can challenge you on this. Here's something that some languages absolutely agree is fundamental and some agree is the opposite, memory management.
00:12:14
Speaker
How do you bridge the gap between a memory-managed language like Java and manage it yourself language like C++? That's a very good question. What I decided to do is to have two calls. You have allocation calls where I use the native one. In Java, I use new.
00:12:35
Speaker
But for the allocation, what you do is that you create a function with a special name called delete. And in Java, it's empty, nothing happens. But when you translate it to C++, it will do an actual delete operation in C++. So the short answer is there is manual memory management. But when you run it in a garbage collected language, it will garbage collect.
00:13:03
Speaker
Okay. And if I'm writing from Java garbage collected language, you are going to automate when you translate it to C++, you will take care of the manual memory management primitives. Exactly. So if you have, in a sense, when you write the code, you have to do manual memory management. But when you run it in a garbage collected language, then you don't, then the garbage collector will take care of it. Okay.
Error management and code consistency in ProgSpace
00:13:30
Speaker
Crikey. But the purity of the language makes this less of a difficult task, actually. So give me an example of something useful you can write in this subset. What are the practical day-to-day limitations it imposes on you?
00:13:46
Speaker
In this subset of the language, you can write a lot of libraries related to doing computations. We haven't talked yet about the library of functions, the libraries of the online library of functions. We need to get into that.
00:14:03
Speaker
Yeah. So I have a large online library of code written in this way. And there we can see what kind of libraries it is. Should I just have a glance at it and give you a few highlights? Yeah, give me an example. So a couple of good examples that I'm using in a project I'm working on is adjacent serializer and deserializer.
00:14:27
Speaker
Okay. That's one example. Another example is... I can see how that would be reasonably global. Yeah. Another example is date libraries dealing with dates, all kinds of calculations with dates. That's actually a huge rule. Yeah. The hard part of that is not so much programming as the crazy rules that the planet has on how they work. Exactly. So here you can capture those rules in a language, which you can then translate to all the languages you want, right? So that you can have the same date functionality.
00:14:54
Speaker
So are you saying that you've written a new date library? Exactly. So you didn't go to an existing Java date library and see if that was translatable? Exactly, but I did not, of course, invent all the algorithms. I look up algorithms, I take inspiration from open source projects that have an open license, mind you.
00:15:15
Speaker
Thank you. I always make sure they have. So I take existing libraries and I convert the code to broad space code, which means removing a few language constructs here and there. And actually, some of the functions get more clear when I do this, because I reduce the amount of features in language I use. And sometimes this pops up interesting questions about how the implementation works. Green example.
00:15:45
Speaker
Yeah, a big example is error handling. I think that's the major one I encounter. Many languages rely on exceptions as an error handling mechanism. And in progress space, you don't have exceptions other than a certain kind of exception where you overflow an array. If you divide by zero, there are certain hard exceptions. But for ordinary control flow, you don't have exceptions.
00:16:12
Speaker
What do you have instead? What we have instead is you return, if a function can fail because the input is invalid, I return a Boolean saying that the function failed. That is always very interesting because when I then start to look into the implementation, I will see, oh, there are all these failure cases.
00:16:34
Speaker
And what you'll see that many failure cases are not handled. And then you'll get this generic null pointer exception, for example. That's a very common one. If you haven't thought about all the errors, I will add additional checking. And what you will get in the end is actually a library that either succeeds at what it's supposed to do, or it fails with the status of false. And I often also include an error message saying why it failed.
00:16:57
Speaker
So if I'm writing some Java in this, does that mean I have to remove my exceptions? Or does it mean that you'll translate my exceptions into? No, you actually need to remove the exceptions. That's true. And go over to this error handling by return value and stuff. OK. So it's trying to get the idea of what it would be like as a Java programmer to convert my code, because it's not going to be entirely idiomatic Java then.
00:17:25
Speaker
No, it's definitely not going to be idiomatic Java, that's for sure. What's interesting actually is, let's say you write some code in this and then you translate it to C++, it will not be idiomatic C++ either. Right. Actually, you will get idiomatic progress space in Java and in C++. The code will look very similar, because one thing I haven't mentioned to you is that the translation isn't one way. It's a one-to-one translation. So that means that the code you wrote will look almost exactly the same in C++.
00:17:56
Speaker
or in TypeScript as the one you wrote. So in some future, I could take the Java file, spit out a C++ file, re-pause that, spit out a Python file, spit out a Visual Basic file, and go for lunch. Exactly. Right. So is your hope here, then, that eventually, I'm thinking probably companies that have a large number of different languages?
00:18:22
Speaker
Netflix has lots of languages, doesn't have any hope of recolorsing those into a few. Are you hoping they would convert their core libraries? That's a good question. One thing I hope to do is that as more and more libraries accumulate here.
00:18:39
Speaker
I hope they will have longevity. And what I hope is that over time, since we do not have to rewrite the libraries in a new language, if a new language comes along, I'll write an output for that language. So it is fully supported.
00:18:56
Speaker
And what I hope is that as the library grows, we get a library that anyone in the world can reuse, not only throughout the world, but also over time, so they can reuse it also in 50 years or 100 years.
ProgSpace's approach to language unification
00:19:10
Speaker
So I hope that this, even though the language itself might look a bit unfamiliar and be a bit controversial, I think the longevity will prove itself in that we get more and more code that is of higher and higher quality.
00:19:24
Speaker
and that that will encourage reuse and they encourage people to use it. Okay, I can see that. So is it fair to say your idea here is rather than trying to find a language to rule them all, you're trying to find a style to rule them all? In a sense, yes. Or to at least harvest what we do agree on now. So instead of creating another faction or another dialect, I try to do the opposite.
00:19:53
Speaker
This is the core in the sense, the core on what we agree on. Let's see what we can do. And what I found was that we can actually do a lot of coding in this language and it works very well. Okay. There are two directions I want to take this in. Let's go into the library space first. So you've been talking about Progs-based the language. When you think about it, are you really talking about Progs-based the spec or the set of capabilities or some internal representation?
00:20:22
Speaker
So should I mention the library, the online library? Yeah, yeah. So in relation to building the language, I also set up an online library. And this is an immutable online library of code. So when you publish a release version into this library,
00:20:42
Speaker
it cannot change in the future. And the library is known by its name. It also has a namespace, and I have two namespaces, actually. I have an organizational namespace, where you put your organization, for example, NO company, or COM company. This would be the organizational namespace. But you can also have what I call a scientific namespace.
00:21:08
Speaker
So if you created a library for doing network, let's say, computation related to networking, you would put that into the scientific namespace, computer science, networking, and then addressing, for example. Okay, this is kind of a side question, but which one do I then import? You can choose both. At the moment, I only have support for the organizational one, but the scientific one will come very shortly. And you can choose which to import.
00:21:37
Speaker
So is that library, is that storing Java code or canonical representation? It is storing the AST, the abstract syntax tree, of what is valid the progspace code.
00:21:52
Speaker
Right. This is reminding me a bit of Unison. Yeah, and Unison is quite similar in this respect, but Unison recognizes functions by their hash. Here, a library is known by its namespace, its name, and its version. Okay. So it's those three, and then in addition, you have a function name, which identifies a function in that library.
00:22:20
Speaker
Okay, so if I attempted to publish a slightly different source tree to the same version number, what would happen? Then you would get a message saying that a version already exists in the library. Okay, okay. And one important thing here is that you also, you bundle with a file where you describe the dependencies of your library. And those dependencies, if you publish a release version, can only be released dependencies.
00:22:48
Speaker
And this ensures that the libraries that are published are always going to stay there as they were. And this whole database is actually available online. So I've created this domain where you can go and then you can browse all the libraries, all the versions, all the functions. You can see the source code in all the languages because I just have a dropdown. You select another language. It will translate it immediately to that language.
00:23:17
Speaker
Okay, so when I'm browsing the source code, there is no source code, you're generating my chosen flavor. On the fly, yes. And interestingly, what I can do is that not only can I do this, I can also generate an applet if you want, or a form where you can run the function there and then.
00:23:36
Speaker
because you can translate into JavaScript. Exactly. So you can run the function right there and then in your browser. So if you upload code to this online repository, that website will be generated immediately for that function.
00:23:52
Speaker
You can also have private libraries, but then that's a separate part. My main focus area is the online open source MIT license source code. As a curiosity, when I download a library,
00:24:08
Speaker
to my local machine. Do I download the AST and store that, or do you translate it for me and I download my favorite language? I could actually put a download link for the AST, but right now I just have download links for each language. So for every library, you can go download and you will get all the languages and different flavors of each language, also different versions of a language, and download the source code. And you will get not only the source of the library, but also all the dependencies bundled into one.
00:24:37
Speaker
So you've had to write an AST pretty printer for all 13 languages. That's exactly right. That must have taken some work in itself. Writing the libraries and doing the research was the hardest part. Writing the language part of it was not that difficult.
00:25:01
Speaker
Then we need to talk about what you call the infrastructure side of things, because at the moment it sounds like it's going to work for just a set of pure libraries. Exactly. So when I was happy with the part of doing pure functions, I started looking into how should we deal with devices of all different kinds. And what I found was that
00:25:29
Speaker
I took inspiration from the capabilities view of when you have capabilities in a function. And let's say you want to interact with a disk. Then you will pass a structure to your function called disk. And then I created three operations on that disk, read and write, and also get the size of disk.
00:25:58
Speaker
These functions are, when you program in ProkSpace, pure. So there are mocks of the disk. So you program it, you completely deterministically, completely pure, and you develop and you test and you see that everything is good. Now you're ready to deploy this interacting with an actual disk.
00:26:19
Speaker
What you would do then is to delete the library called disk, which you have imported, and replace it with, for example, a file disk handler that will do these operations on a file on their computer. So then it will treat that file as a disk.
00:26:36
Speaker
Okay, so you're enforcing, I can't do side effects, I have to program against an interface. Yeah, exactly. And I've selected many devices, but let's see. I've selected six main devices that covers most of the productive things you want to do.
00:26:57
Speaker
Give me the six. Give me the list. Yeah. So there's a disk. There's a disk, and then there's a screen, putting stuff on the screen, a clock device, audio device for outputting audio. And then there are two where I needed to invent the name, because there was no standard name for them. And I chose to call one a processing unit, which basically means a thread. It's a single thread, memory, and CPU. OK.
00:27:25
Speaker
And then a processing unit controller is what I call the last one. This is where you want to start or stop a thread. Okay, so it's a bit like a thread pool.
00:27:36
Speaker
In a sense, yeah, because the processing unit controller will have, let's say, six processing units you can control, and you can ask any of them to stop or start, or you can change the program that is running on them. And this you would use to create a scheduler, for example, or if you wanted to split one thread into multiple threads to do. It's a bit like a supervisor then. Yeah, yeah. Missing from that list quite notably is network. Yeah, this is a very good question because
00:28:05
Speaker
I have abstracted the idea of a network and combined it with the multitasking. So when you write process-based code, it's always running in a single thread. And if you want to run two threads, either on one computer or on two separate computer, the only device you interact with is the processing unit device. So you will pass a processing unit device to your function. And then you have
00:28:35
Speaker
Yeah, you have a number of calls, but basically, if you're interacting with this processing unit as a client, you can send, you can receive, and you can also check if there's a message. Okay. But you can also interact with this processing unit as a server. And then I would have server receive and server send, and also there check if there's a message available.
00:28:58
Speaker
So if I spin out a local thread or I treat someone else's web server as a thread, it looks the same. Exactly, exactly. It only depends on what kind of driver you use for the processing unit.
00:29:11
Speaker
On the same computer, you could typically have a shared memory pool. That will be the most efficient way to do multi-threading on a single computer. But the code would be exactly the same if you wanted to interact with another computer. Then you would replace the network call, typically a TCP call, for example. Does that get tricky when you're doing things like serialization? How does that play out?
00:29:40
Speaker
I'm uncertain what you mean. Because I mean, most threads you're just sending data to directly. Sometimes you're even sharing a pointer in memory. Whereas now you've got to worry about writing bytes to the network. You've got to do more serialization work to make those calls. Yeah, you're right. So that's a fair drawback, I think. But here you can have a unified view of whether you work on a single computer or multiple computers.
00:30:07
Speaker
So what's actually happening under the hood? Are you serializing every call, whether it's local or remote? So the calls you have against the processing unit is send an area of bytes and receive an area of bytes. So it's up to the person who makes the code that decides how to serialize it. But there is actually a way to optimize code in Proxspace that we can talk about. I don't know if you want to talk about that now or later. Let's go for it. Let's go for it.
00:30:35
Speaker
Okay, so we actually get a huge benefit from the fact that these functions are pure and that there's no global states.
ProgSpace in production and interfacing with systems
00:30:44
Speaker
This means that we can actually replace individual functions with a more efficient call. So if you wanted to pass the pointer directly, you could write a more high-level call and then replace that with a driver doing that actual call. Take me through that.
00:31:06
Speaker
So before we dig into that, maybe we can talk about file systems just to have an example. Most people think about the file system as interacting with a disk. But the fact is that in Progspace, if you wanted to interact with the file system, it would be a processing unit. And the reason is that a file system is actually a lot of logic that needs to be done in order to figure out how to talk to the actual disk.
00:31:35
Speaker
So, if you had the file system called read file, for example, that would be an API call to a program implementing a file system.
00:31:50
Speaker
But you could actually, if you wanted to, replace the function read file with a more efficient one, especially tuned for Linux or Windows. And since functions give the same output for the same inputs, this would actually work. It would be easy. Because in C, for example, or C++, you can't just replace a function because you don't know the effects of what it will do. But here, since you have this very limited, very tight
00:32:19
Speaker
tightly focused language, you can start to do these kind of things more naturally. Why not unify disks as though they were threads as well? Because you just file system, you send stuff to it, you get stuff back from it. If you're going that abstract, it looks a lot like a network call or a thread call.
00:32:44
Speaker
It does, but a file system usually contains other things like a file name, it contains access rights. You can also get error cases like the disk is, like there's no more... Specializing the error. Yeah, so it is a considerably larger interface, a file interface. Okay, so it could be a subset, but you've just chosen to make that more fully featured. Yeah. Yeah, okay. Okay, I can see that.
00:33:06
Speaker
So yeah, what I find very nice about this is that you have a unified view of multiprocessing on a single computer and over many computers. So this means that it is quite natural now to create a distributed program running on many computers. To what degree have you done that out in the wild?
00:33:27
Speaker
I've done that. I started developing these libraries a year ago, and they're expanding quite rapidly now. So I now have a number of these kind of modules building blocks for these kind of distributed systems. Should I mention some of these building blocks? Yeah, yeah. Yeah, go for it. They are nicely categorized into these scientific namespaces that I mentioned for you. But let's see. So we have a load balancer, for example.
00:33:53
Speaker
And that is a program where you take a number of clients and a number of servers, and it deals with the load balancing in the tweet. And that can be applied either you want to load balance across servers, or if you want to load balance locally on your computer, or you want to load balance partially on your computer and partially on another computer. It only depends on what kind of processing unit drivers you set up when you run it.
00:34:19
Speaker
So yeah, I'd be able to expect to run a load balancer with a number of different servers behind it. And one of them is running the code in Python and the other one's running on a JVM and the other one's running on raw binary, what else?
00:34:36
Speaker
Yeah, that would be natural to do here. Yeah, that was something that would be natural to do using the setup. But the nice thing about this is that you can actually, you can start work as an architect, you can think of, okay, let's now consider what kind of modules I want in my system.
00:34:51
Speaker
I want to have several disks, and then I want to have a RAID component that does the RAID logic. And then I want to have a cache, so you put a cache module. I also have that in the library, a cache module, a RAID module. And then I want to have my file system, and then I want to have another layer of cache on top. Or you can just remove the caches and have a simple infrastructure. So you can start doing this kind of architecture where you just select the components you want.
00:35:17
Speaker
and build them and run them. And because the interfaces there are so simple, you can also run it on a bare metal server if you like. And I've been working together with the bare metal OS to get the drivers going for a bare metal deployment. Okay, what stays that in?
00:35:37
Speaker
It's a good set where we are working on the networking components. That's the only part that is missing. But inter-process communication, disk access is always very natural on a bare-metal environment. But you don't need to run this on a bare-metal environment. You can run it on Linux. And then you just use the Linux drivers instead. You use the Linux driver for TCP communication, the Linux driver for disk access. So this kind of working, this kind of architecture, I've coined a term called No OS.
00:36:06
Speaker
because you are not dependent on any specific OS to build and run these components. That means that you are in very good control over what is happening. So I've got a chance then of expecting to run some code on the JVM and on some embedded hardware and on AWS Lambdas potentially. Yeah, actually possible.
00:36:34
Speaker
Okay, we need to talk about how far along this project is. Is anyone using in production? So, I have not done a lot of marketing on this project. This project has been going since 2018. I have been steadily building updates on the language stabilized around 2019. And I have started building, I've built libraries over many years. And then the last two years, I finalized this capability model. And I started building capability, these kind of drivers and infrastructural components.
00:37:06
Speaker
So in the fall, I consider now the project is pretty complete. Now I want to do some marketing and get this out there. So I'm very early. But I run this company here. I'm in the office now, which is a cloud provider company in Norway. And we have been using this since about 2019. We have been using many of the components I talk about in production. OK. For what kind of tasks? For many kind of tasks. I mentioned some of them earlier.
00:37:36
Speaker
all kind of data calculations. We have serialization using JSON and Base64. We are facing it in now as a static web server. Because we have all the components we need to do static web serving. We'll be facing in load balancing shortly. And we'll also be facing in a kind of RAID operations
00:38:04
Speaker
and stuff like that. And they're also facing in database logic as well. So it is slowly facing in many components. Database, is that going to be written in Progspace? Yeah, exactly. We'll have libraries for different database operations. The one set up at this point is a key value database, but the relational database is also in the works. If I wanted to talk to Postgres, for example, what's going to happen there?
00:38:33
Speaker
Yeah, so if you want to interact with some kind of software that is not written in Progspace, what you would need is an adapter. So you would need an adapter that takes, for example, an SQL request as text, and then passes it on to Postgres. So that's how you would actually interact with a system like that. You would regard it as a processing unit inside Progspace. Okay, so...
00:39:01
Speaker
I would then presumably need to manually write that adapter in every target language that's on me. No, I don't think you would need that. Maybe parts of it, but the main parts of it you could do in Prospace and then maybe a tiny part of it needs to be done in the particular language.
00:39:18
Speaker
What about external libraries, though? If I'm writing in C, I'm going to bind to the lib postgres client. I can't remember the exact name. That's not going to work in Java land. Yeah, exactly.
00:39:35
Speaker
This whole basic setup that I've been talking about only works if you're doing everything in Prospace right. So that is the limitation, absolutely. But my hope is that the benefits of this language means that if we build certain components using this language, we will get certain features and maybe some parts are outside.
00:39:57
Speaker
But my hope is that we can get a database of libraries that are very solid, that can be used in any language. And if you have some task that is not suited for that, then sure go ahead. One benefit here is that you can only import a progress-based library into your project, and it doesn't influence any other part of your system, right?
00:40:18
Speaker
Because it's just more Java or whatever language I like. Yeah, exactly. So it's very easy to reuse the components as they are in any project.
Testing ProgSpace's robustness and reliability
00:40:25
Speaker
So if you want to use progress libraries in your project, it doesn't limit the project. It's just a Java file that sits there and does its thing.
00:40:34
Speaker
And you can do whatever you want with the rest of your project, of course. But if I happen to want the same library in another language I'm using, I can expect exactly the same behavior. Exactly. So hopefully, that is a huge benefit. And if you want to write, as I said, infrastructural components as well, and have that as a part of your system, that works as well. But of course, you're entirely, this doesn't dictate how you do all your stuff. So out of curiosity, when you phase in your load balance at a production,
00:41:04
Speaker
Presumably your target language isn't going to be visual basic. That's true. Which one will you pick? I have a tendency to run most of my stuff on Java, on the JVM. But if I wanted to have more speed, for example, I would put it in C. So some of the components are in C just to get that additional speed.
00:41:25
Speaker
That'd be really interesting to benchmark, given that you're pretty confident it's exactly the same code. Yeah. And we haven't talked about this yet, but one hope I have with these libraries is that they will improve in quality over time. Because what I try to do is to have libraries with a single thing that they do and then do it well.
00:41:47
Speaker
And then I've created this process where I will hopefully evolve this library into a very robust and high-quality component. And my idea is that I start with creating a component. Then I put it into low-importance production work to check what the quality is. Then I put it into more serious production workloads.
00:42:07
Speaker
And when it's been there for us some time without giving any error, any unexpected behavior, any errors, it works as it does. Then I put it up on prospects.com on what I call a bug bounty program. Right, I have seen this you've got your most confident libraries, you're putting your money where your mouth is. Yeah, yeah. So because
00:42:29
Speaker
When I've written the library, I see it worked in production. It has done so for some time. Now I put it up on my line and say that you will get, for example, $200, $100 for finding a single bug in this library. And you might say that's a bit overconfident, but I challenge anyone viewing this to go and see if they can find bugs. You can actually run the code just in your browser. There are links you can run. You can try any output you want. You can analyze the code with your favorite faster.
00:42:59
Speaker
And if you find the bug, great, because then we don't have that bug in production anymore. So that would be absolutely fantastic. And that's actually worth the money. And if a bug isn't found over some time, I will raise the reward. OK. So that's just any bug for the library, no matter how large or small. Yeah, exactly. As I say in the bug bounty program, if you can find a valid input that gives you an unexpected output, that's your reward.
00:43:29
Speaker
What if, does that still count if I only find it in a corner case in Visual Basic? No, so the bug about the programmer explicitly say that it has to be...
00:43:42
Speaker
Yeah, if you find a bug in specifically for Visual Basic, that's a more internal error in our progress-based system. So we don't have a bug bounty yet for that. But that would be an interesting separate thing, actually. But as of now, it has to be when you run the code in the core sense. It would have to be present in multiple languages. But we have a bug in the library and not in the writer itself.
00:44:11
Speaker
Yeah, that makes sense.
Getting started with ProgSpace
00:44:12
Speaker
I could see it's kind of a next order of magnitude abstraction to test the entire language translator rather than the library. But I think you made a very good point that if there is a bug in the writer, that's also quite serious. So that is something I actually could put up as a bug bounty as well. That's a good idea. It's a separate bounty. Exactly, exactly. Good idea.
00:44:35
Speaker
OK, so in that case, if someone wants to get started trying this, if I fancy kicking the tires on a Java to Visual Basic to Python translator, where do I get it? Yeah, so I have a website, Proxspace.com, and you can go there and you can download the tool. It's called Proxspace. You can run it on the command line.
00:44:54
Speaker
Because this is an online library where we have the code, the tool interacts with the online library. So therefore you need an account, but the account is free. And as long as you just upload open source projects to the library, contribute to that, everything is free. And if you want to use Proxspace libraries in your system, you can also use the Proxspace tool to import the dependencies.
00:45:20
Speaker
And you can do that in all the languages we support. And if you have a language that you would want to import the code in, just let me know and I'll see if I can make a writer for that language. Okay. So if I then go away and write some Java and I decide I want to make it into Python, do I have to upload it to your library server and then redownload the Python?
00:45:45
Speaker
No, because the tool has a command called converge, and then it will convert the code. So it can run it all locally? Yeah. Yeah. Okay. That makes it very interesting for like little local experiments on language to language translation. Yeah, it does. It does. And you can also go on the online repository and do the drop-down menu. And this online library is also highly interactive with that. You can click on a function and you will get to go to that function and you can try to translate that as well.
00:46:13
Speaker
Cool. Okay, last question then, what's the most exotic language I can translate to? Let's see what the list is. Yeah, maybe the most exotic language is VBA.
00:46:27
Speaker
VBA. Yeah, that was a pretty exotic one. But it's an interesting one, actually, because it's used in a lot of businesses. Yeah, they don't brag about it, but it's actually out there. I can see that being surprisingly popular to you can write VBA without actually writing VBA.
00:46:44
Speaker
Yeah, so here you can set your developer to create a large library. He can test it locally using his favorite Java ID. And when he's happy with it, he can upload it to the library and download it as a VBI library. OK. I could see that being a little niche corner of your user base one day. Yeah, but your question was what was the multi-exotic? Yeah, yeah, yeah. I love that though. It is exotic, but you're right. It's still desperately used.
00:47:13
Speaker
Cool. Okay. I think I have to go away and play with this. I'm going to give it a try and see, uh, see how far it goes in translating, in being like a Babel fish of programming languages. Martin, thank you very much for talking to me. Yeah. Thanks for having me. Have a nice day.
00:47:32
Speaker
Thank you Martin. You know, if I were Martin's marketing department, I'd say that last bit might be the best argument for adoption. It's quite compelling. Tell people you can write the language you like, but emit the language you don't. We all have feelings that way, don't we?
00:47:49
Speaker
I don't know, it's an idea that needs some battle testing though. I would need to kick the tyres before I formed an opinion. Let me know what you think in the comments. If you think it's a good idea, go and check out Progsbase.com, link in the show notes as always. If you think it's a bad idea, you should probably go and check out the Bug Bounty. Because if you're right, then that's free money. And if Martin's right, it's a free education. You're a winner either way, right?
00:48:15
Speaker
Before you go and do that, please take a moment for some feedback. If you liked this episode, click like or rate or whatever buttons your app has. If you know someone else who might've liked it, please hit the share button and send it to them and make sure you're subscribed because we'll be back with another episode next week, of course. Until then, I've been your host, Chris Jenkins. This has been Developer Voices with Martin Johansson. Thanks for listening.