Coding a text editor in Odin and Raylib on Raspberry Pi 4 that talks to Odin Language Server

[00:00:00]I mentioned in my last video that I was somewhat unhappy with the sluggish nature of VS Code or VS Codium on my Raspberry Pi 4 here. So, that got a little bit of a bug in my mind to work on my own text editor again. And I haven't worked on a text editor since, you know, I don't know, 3/4 of a year ago or something. So, why not? Only this time I'm coding in Odin instead of Go.

[00:00:25]And I'm using Raylib for the visual rendering of it. Let's kick this off.

[00:00:30]And we see it kicks off pretty fast, which is nice. Now, unfortunately, my text editor doesn't have all the features, like it doesn't edit text yet, or scroll, or various other things. But you can move the cursor around with the mouse pointer or with the arrow keys.

[00:00:47]And it also talks to the Odin language server. So far, just for hover help text. Only I'm purposely not hovering.

[00:00:56]I'm purposely putting the help text or anything else on the right-hand side so that the primary code editing area is just editing code and nothing getting in front of you or getting in your way. So, for example, if I mouse over in VS Code or Codium again, I get all this help text in the way of the code I'm trying to edit. So, that's a purposeful design decision that I've been differently from other standard editors out there. Now, wrapping the text or better markdown rendering could be nice in the future as well, among other important features like editing text. I have a long history of abandoning hobby projects, but if I keep this thing going, the idea would be to continue this philosophy in the user interface.

[00:01:40]I also have way too much logging happening right now, which obviously should be cleaned up in the future also.

[00:01:47]Now, I also have a separate script that can build and run, which is relatively straightforward in Odin, but I made a separate helper script anyway.

[00:01:55]And we see that it builds pretty fast even on the Pi 4.

[00:02:02]Taking about 4 and 1/2 seconds here. I think it's actually usually faster than that for me. I wonder if VNC is taking up enough CPU it's slowing down some.

[00:02:09]I'm not sure. And the built binary is about 2 and 1/2 megs or a big chunk of that is just the font data that I'm including in the binary. My editor also uses very minimal RAM and when you're not doing anything on the user interface the GPU and CPU usage goes down fairly low. I'm trying to keep all that stuff in mind whether or not I continue the project later. Meanwhile, let's move off the Raspberry Pi because VNC makes the interaction a bit sluggish here. So, I'll switch to the main computer I'm recording from. Even though again vast majority of my actual development for this editor is being done on the Pi 4 itself.

[00:02:45]So, back to Codium again and the same file we've just been looking at on the Pi 4 and even including in my Sprite text editor. I have a command line option for tracking memory allocations and deallocations. Other than that, it knows how to see if you specified a file to load and loads it and cleans up when you're done.

[00:03:04]And there's our binary bundling of the font data into the editor itself. And part of the review of the code here will be my review of Odin as I'm going since this is my first time trying to write anything substantial in the Odin programming language. High level summary is I do enjoy using Odin. I'm also glad that the language is relatively stable at this point. So, let's do a quick tour through the code not hitting every detail.

[00:03:28]So, here's my app. I have editors and language clients. Where I've coded the language client support myself but I have been referencing Odin language server while doing that. And I'm really sort of winging things as I'm going because my hobby time is limited.

[00:03:43]I haven't explored built-in regex from Odin but I'm not using it yet. It could be this editor goes somewhere PCRE tool will work better than Odin's built-in regex.

[00:03:53]Meanwhile, doing lots of raylib here and setting up a single text editor for the code side and a separate one for the help side.

[00:04:03]I've got a destroy here and I find overall I can keep track of my resources in Odin fairly well. I am using multiple threads for the language client, which we'll get to in a second, and maybe I could have used non-blocking IO instead, but I've done what I've done. And dealing with memory across threads is harder for me to reason about than single-threaded handling in Odin, and maybe I just need to learn my idioms better.

[00:04:28]Here's the main loop. I'm purposely targeting 30 FPS, although I can render faster than that.

[00:04:34]And typically, if there's no changes happening, I'm actually skipping the render.

[00:04:39]But having a 30 FPS tick makes it easy for me to think about timing stuff later without worrying about seconds and so on.

[00:04:48]And then I'm mostly just drawing stuff for the editor and the help text.

[00:04:53]I'm also updating my language clients.

[00:04:56]And we'll dig in more into the language server client here shortly, but we can see here that we're talking to our client, actually clients. I only have one really. There's probably some sloppy stuff here I need to fix up later. But I'm checking to see if the hover I get is for the help I'm expecting, and if so, then I'm updating on the right-hand side if it has changed. Other than that, I have to tell Odin language server or OLS that I opened a file, otherwise it doesn't recognize anything I'm asking for later.

[00:05:28]And then I also have to send hover requests for whenever the cursor moves around on the screen.

[00:05:34]I do check to see if there's no change in my request or if I'm within the range of whatever it told me last time that I don't make a new request to the server.

[00:05:43]And when I'm loading a file, I have to send that did open like I said. Okay, the document inside the editor. And actually, maybe I should look at editor first here for a second. I have the text data inlined directly into the editor. I don't have a separate here's my model, here's my view going on. I started going down that path and thought I actually don't care about that for the scope of what I'm working on.

[00:06:06]I don't think I care right now about having multiple views of the same document, even though that gets used. I can do split screen, for example, in VS Code here. But I'm concerned that flexibility would encourage a more complicated editor experience.

[00:06:20]So, I chose to simplify down to where one editor has one document. And if you want to change the document in the editor, you swap it out if you want to.

[00:06:28]Or if you want multiple editors, you can. But I don't want to have multiple views for the same document data right now is my current idea. And not aiming for a collaborative text editor, either.

[00:06:38]So, the document is just a dynamic array of lines. Lines have spans on them.

[00:06:42]Right now, I actually only have one span per line. This is me thinking about how I could do text wrapping, but I haven't really implemented it properly yet.

[00:06:50]And I'm imagining very simple list of possible things that each span, each portion of text on a line could be for syntax highlighting purposes, which I also haven't implemented yet.

[00:07:02]It's all very simple stuff.

[00:07:04]And I'm also primarily using the main context allocator for things that outlive a tick on the screen. So, like anything about requests to the server or the document itself have to live on the main allocator. Things I'm allocating that live just for the current tick of the updates and views, that stuff I'm putting on temporary allocators.

[00:07:26]But I do make sure to free old stuff before loading a new thing, for example, for the help text.

[00:07:32]Now, for drawing, I do one thing that's probably too complicated. I'm using DrawTextEx from raylib, which means I need to pass it a C string. And most of my text does not have C-string conventions. I'm using standard Odin strings, which have length and string data and aren't focused on being null-terminated. So, I'm making sure that each segment here, which could be actually be a span in the future and not just a line as a whole, I need to make sure that anything turns into a C-string. And because of that, I actually do special custom handling just around this to make sure that I'm reusing the same space and getting rid of it after I'm done with the document draw. So, sort of a customized temp allocator in a way here. But, I might change the drawing just one glyph at a time in the future. And if so, then I don't need to make a separate C-string, and I might throw that part out. And while I'm saying glyph right now, it's mostly code point. But, if I saw correctly, the Odin standard library actually has multi-code point glyph processing inside of it. So, I could possibly get fancy with my glyphs than I'm doing so far without too much effort on my part.

[00:08:35]Again, the editor itself has both edit state as well as a document. I'm actually writing it for multiple selections because I've started using selections a lot in my editing inside of Codium, but I only have one selection at most for the moment.

[00:08:50]And here's lots of things for the future. And as for editor itself, I'm doing things where I'm calculating an input state separate from the raylib input functions that I have available.

[00:09:03]Because once I get an input state available, I can say whether things have changed or not, whether I need to do a redraw or not.

[00:09:10]But, importantly, something I'm doing, and I got this idea from Ebiten Engine in Go, is actually keeping a tick count on how long keys have been pressed.

[00:09:19]Because that makes it much easier logic to deal with repeating things.

[00:09:26]And that's the gist for the moment.

[00:09:28]Okay, moving on. Also, you get moving around the screen with the arrow keys. I have this idea of key active here for handling my own logic for repeat on holding down arrow keys. I probably might not have repeat for holding down text. We'll see. But meanwhile, I want to have an immediate response on the first tick for when an arrow key is held down. Or if you've held it for at least half a second, I want to have a repeat on every tick. So, let's see how this works.

[00:09:56]Let's go and build and run it. Now, Odin builds fast enough that it's not really a big deal. And that actually is a little bit of a delay perhaps from not being fresh. I don't really spend time waiting around on things. Even like I said, it's only a few seconds worth even on the Pi 4.

[00:10:10]So, if I hold down an arrow key, for example, we can see it handling that tick just like we said.

[00:10:18]When you hold off and start again, there's that half second delay, and then every 30th of a second it comes along again.

[00:10:24]And here's OLS working on Windows as well.

[00:10:28]And I do love that responsiveness.

[00:10:29]There's a tiny bit of a delay there, but it's really not bad. I quite enjoy it.

[00:10:34]And here we got way too much logging again on Windows also.

[00:10:38]So, there's how easy it is to implement that kind of logic here using this tick notion. Which again, I mostly borrowed that idea from Ev Engine.

[00:10:46]Okay, let's do a quick skim through the actual language client itself.

[00:10:51]And I implemented this myself, though somewhat looking at both the specification and also looking somewhat at Odin language server source code. I'm actually really concerned about only a handful of languages in my text editor.

[00:11:03]I would prioritize Odin, Lua, GLSL for shaders, and markdown as the primary languages I would care about. And so, language servers for those languages are the ones I would need to implement enough to get the job done for whatever features I want. Sort of the opposite job of the language server writer who thinks, "Which editors do I need to make sure work for my language server?"

[00:11:26]For fun here, let's just run the language server directly, OLS.

[00:11:31]So, to understand how language servers work, we just start up a process and we talk on standard IO for everything that happens after that.

[00:11:39]This log message is just them being nice. I have to actually jump through some hoops before I can actually start sending things like cover messages. I have to actually go through a little bit of a handshake of me telling them things and responding to what they tell me in order to get things kicked off. And I can't usually do that by hand, not only cuz it's no fun to write JSON by hand, though I could be copying and pasting it. But also this header section here, where I have to have a content length, and this is carriage return new line after each of the headers here, just like for some other protocols out there, which is annoying and awkward for just copying and pasting text. So, I haven't worried about how to make that work or if it's easy or not. So, we'll just look at my code that's doing the protocol instead.

[00:12:24]So, here I'm starting stuff. Wow, that's way lots of kickoff. My language client has channels for communicating across threads, as well as standard in and out from the language server for talking to it, and keeps track of the process itself, so I can like kill it if I want to or things.

[00:12:41]And I'm planning to keep track of all outstanding calls in a very simple fashion here. And something central has to know whether the ID I sent out is the ID I'm receiving to know which call it goes with. So, while I have separate threads for reading and writing, there's a central logic here on the main thread that puts everything together.

[00:12:59]Okay, there's the init. Lots of busy code.

[00:13:02]Here's the destroy, which I'm actually not doing a great job of. I've actually tested and mostly my memory is clear, but I don't do fully proper cleanup yet. Something I could improve in the future, like when from the app itself, the whole thing is over.

[00:13:17]There's my interaction with the threads and sending some stuff. Okay, let's look at the requests here.

[00:13:23]These are requests for sending to the language server, which means this is what I'm writing out and what it's reading in. And Odin has built-in JSON marshalling, so I don't need to worry about it very much. It has unmarshalling also, but there's a complication with that that maybe is easier than I thought it was, but I worked around it manually, which maybe was silly of me. I didn't even try to see if it worked automatically.

[00:13:44]But, I'm building these structures here specifically with the right naming conventions so that JSON marshalling is direct and automatic for the language server protocol. That's why I'm using camel case instead of snake case here, for example.

[00:14:00]Okay, cool. There's my requests.

[00:14:03]And here I'm using the generics capability of Odin, which is actually rather pretty.

[00:14:08]And so things get written out here.

[00:14:09]Here's JSON marshal. It just worked for me so far. No big deals. And again, for any one particular thread, life's pretty easy. So, I can do a temp allocator allocation for my JSON message, send it across the wire. I don't care what they do with it, and for me it'll just get freed up on the next temp allocator clean for this thread.

[00:14:30]Here are the responses.

[00:14:33]And these are different. These I'm actually not unmarshalling automatically from JSON because I have unions to deal with, and so far for this union, I've only supported the things that I see OLS doing. But, I don't know how the Lua language server will act in the future if I get around to it.

[00:14:51]And I've designed these things for the idea I'm manually going to figure out which version of a thing I'm getting from the language server.

[00:14:58]Here's the read logic. Okay.

[00:15:01]There's some initial setup here that's for initialization. Otherwise, we're going to be reading JSON objects up here with JSON.parse.

[00:15:12]And then, assuming we've initialized everything, we're mostly just sending that object demarshaled into maps and dynamic arrays and so on in Odin across the thread or managing it on the main thread later cuz I need to tie the IDs again together for what I'm expecting across threads. And so what I've allocated on this thread, I'm having to deallocate on a different thread.

[00:15:35]And then here's where I'm processing that. If the ID is the hover ID, then I process or parse I've a bit of sloppy nomenclature here the handling of the hover. I see is it a JSON object? That's what I expect. I expect certain contents in it. You know, so I parse those hover contents.

[00:15:54]And I'm checking to see specifically what kinds of fields it has to see how I want to organize my representation of that hover.

[00:16:02]So if we look over here at the LSP specification, we see that union of marked string or an array of marked strings or markup content.

[00:16:11]And you have to know what's what based on the fields that are present. There's no type markers on these things. And I didn't know if JSON on marshaling in Odin would be able to figure these things out automatically. So I actually didn't even try.

[00:16:24]Someone can tell me if that was silly on my part.

[00:16:27]And so that's the basic gist. It eventually gets back over to the tying together of IDs and then eventually also gets back over to my handling of it from inside the app part of things and it shows up on screen.

[00:16:43]Again, the build and run is actually pretty fast here.

[00:16:47]And let's go see what size that was on Windows. I'm getting about hmm two and a quarter megs on Windows versus the two and a half megs I was getting on the Raspberry Pi.

[00:16:56]And so that's that.

[00:16:58]And I'm pretty happy with things. Like I said, I've done some memory checking with the tracking allocator. I've done some with address sanitizer. I've done some with Valgrind and I'm pretty happy with managing my memory these ways.

[00:17:12]And again, the juggling of it across threads wasn't super fun for me so far.

[00:17:16]I sort of miss that automatic management of things I expect from something like Rust, but beyond that I actually enjoy writing in Odin more. I enjoy the zero and net. I enjoy I can print out structs whenever I want to and not think about it. I enjoy that struct equality just works out of the gate. I enjoy just everything feeling lighter weight instead of having to jump through hoops and dot eyes and cross tees all the time. Again, except for the management of stuff across threads really is the biggest complication, but there are tools for dealing with that. And again, I don't know if I'll continue this editor in the future, but it sure could be fun and I like having a lightweight editor that's not sluggish on my Pi. But I do have another distraction that's gotten my attention already, which is this beautiful Adafruit Fruit Jam with an RP2040 microcontroller inside of it.

[00:18:02]And it only has half a meg of core RAM and 8 megs of extra RAM sitting on it.

[00:18:08]And it's even a little bit slower. And going back to the Raspberry Pi for a second, I talked about my last video that getting by on 4 gigs of RAM felt extravagant on my Pi 4. Maybe I could get by on 2 gigs. Well, for the Fruit Jam, I would like to figure out how to get by on either half a meg or 8 megs.

[00:18:26]Now, this is means like a thousand times less than I'm talking about here running on Linux. And the Fruit Jam doesn't need its own OS and that's part of the fun of it. So, here's just a bit of a spoiler of something I was looking at that might show in my next video with an eye toward running code efficiently on the Fruit Jam. Anyway, if you like the video, be sure to subscribe. Bye, y'all.