Emacs: Completion in the buffer with completion-at-point-functions

[00:00:01]Hello everyone, my name is Proilos also known as ProR. In this video, I will demonstrate how Emacs performs in buffer completion. I will tell you about the back end, how things work behind the scene and I will also introduce you or demonstrate uh some of the various options that are available for displaying the results for controlling the patterns of interaction. meaning how we can select one of the possible suggestions. What I have here in front of me in my Emacs is the scratch buffer and below it a text that I have written in org mode. It's an article that will elaborate rather that elaborates on what I am about to demonstrate. I will publish this on my website and include a link to it in the description. So if you are watching this on the video hosting platform, you can follow that link to my website protoillars.com where you can read and you can also check the sample configuration that I have for all the relevant options and packages that I will cover in this video. Let's start with the back end. Okay, so there is a variable called completion at point functions.

[00:01:20]This is a variable that accepts a list of functions. Technically, it's an abnormal hook. Now, we don't need to go into the technicalities. Fundamentally, it's a variable. Its value is a list.

[00:01:33]And in that list are symbols of functions, names of function. Okay. So when you want to perform completion in Emacs such as if I type here something actually let's see first the value of completion at point function. So right now the value is a list that consists of these symbols and we will go into that.

[00:01:56]Now if I want to perform completion in an EMAC buffer, I have typed something and I will invoke the completion at point uh command or in this case I will just do it with tab and Emac will provide some suggestions. It will read what I have typed and then it will consult the completion at point functions where it will run all the functions in the completion at point functions from left to right until one matches. Left to right means from first to last. So it will run them one by one.

[00:02:31]Whichever returns results is what you will see here. And the AMAX will stop there. It will not go through all the function. It will stop there and you will see the results. And now either you have to keep typing or you have to select one of the available options and you can keep tabbing like that and then you do the results. What you are seeing now here this interface is the absolute default interface. This is the completions buffer and is what you get out of the box. I will go into the front ends the patterns of interaction a bit later but for now let's h cover the core the back end the completion at point functions. So as I said this is a list of functions this list of functions can have a global value but will typically be uh local to each buffer. So there will always be a buffer local value in places where you would normally want to be performing some kind of completion.

[00:03:33]For example, I am now running the list interaction mode in this scratch buffer.

[00:03:38]And you see that the value of completion at point functions right here is a list that consists of uh two elements. One is the e-list completion at point. That's the name of the function and the name the second one is the letter t. We will go into that. Let me one second go to the bottom where I have my org mode file. And now I will be like MX describe variable the completion at point functions over there. And it will tell me that here the completion at point function has this local value which I have highlighted right here. And there exists a separate value which is the global value which is what I have highlighted in the line below. Okay, this is the idea and again just for completeness let me do it here also with a describe variable. its value is this and then its global value is that its local and its global value. Okay, this is the idea. Now, why would we want to have buffer local values as opposed to one global value to rule them all? The answer is that we want to be running functions that are relevant for the mode we are in, for the context we are in.

[00:04:59]Here I am running the list interaction mode meaning that I am doing some EMAC list programming. As such it makes no sense for me to be running functions completion functions that are unrelated to Emacs list. Like why would I want to have for example some Python related function running here? That's useless.

[00:05:21]It's wasteful. Right? So I want to particular particularize the completion at point functions such that they work that are optimal and they work in this context. Same principle when I am writing something in org mode why would I want to have all the programming related completion at point functions when I am writing pros there when I'm writing pros I want something that will augment my capabilities of writing pros that's the idea okay so that's why we do it like this now you may notice the letter T here and the letter T here is not an actual function the rest are actual functions So here if I do describe function the e-list completion at point I can learn more about it.

[00:06:08]Fundamentally it is a function that takes the text I have written and will uh take it and perform some computation against some table that it knows about some data that it is aware of and it will tell me okay what you have matches this so select one among those. Okay that's basically the idea. We don't need to go into the technicalities. But the letter T here the letter T is not an actual function is just a special kind of non-nil value. What it means in this context is that I want or rather I emac will h go through the list of completion at point functions that are local to this buffer and once it encounters t this is a placeholder for the global value. So it means once you have exhausted all the local values and you have encountered t go to the global value check what is the global value okay and then of course run the functions there one by one again until one returns results at which point show me the result okay show me the results is the part where the front end takes over and this is what I want to talk to you about so the back end is the completion at point functions it is the modern Emacs way of doing things. It is the current standard. I believe it has been the norm since Emacs 27. Since Emacs 27, a lot of things have happened behind the scenes for completion to be robust, for the framework to be uh fully fledged and for everything to work the way we want it to work. So if you are using something that doesn't use completion at point functions nowadays you may want to review that completion at point functions is the new normal. It has been the case for a few years already and the packages that I will uh mention now build on top of this uh core functionality. Okay. So now let's get to the front ends. I said that when you request completion, Emacs behind the scenes will call the completion at point functions with the text you have written. It will do some computations, some magic and then it will give you the results. At that point, the front end takes over. So in this case, if I request completion, the default front end will show up. It's the completions buffer. And by default, the completion buffer has an interface that maybe leaves something to be desired. I think it's not that good out of the box. It has a lot of options. It has been greatly improved in recent versions of Emacs. So, you can customize it heavily to actually be a very decent interface, but out of the box without any customizations, I think it's not that good. Okay. But the point is that it gives you a view of the options and then you can uh either use the key bindings which you see here or you can use the mouse and pick an option and you have that completed here. Let me just pick something here for example like this.

[00:09:20]Enter and it completes like that and then you continue with your work. Okay.

[00:09:25]Fundamentally it is meant to help you type in long sequences of text faster.

[00:09:32]That's the idea. And of course, if you are working with um language that you don't know very well or in a codebase you are not familiar with, the completion at point functions you will have can help you also be more productive uh with your work there. Not just complete text faster but actually discover functionality and that sort of thing. Now let me show you another uh front end which is company mode. I will enable it now. So company mode is a package that has been around for uh several years now. I believe it has been around from before Emac 27. So in the premax 27 days before completion at point functions became the standard. Uh there were different providers for the equivalent of completion at point functions. So different backends to perform completion. Okay. And company provides several of those back end. It has a legacy of shipping with many of those for different programming languages or different kinds of data.

[00:10:39]Okay. But at its core, company is the front end. So for example, now I have enabled company and when I request completion, I see a popup show up. There is uh in development as of this recording now there is in development uh company version 1.1.0 zero I believe which introduces several new features. I don't want to go through them but basically company even though it's a relatively old package it has been enjoying continuous development and it has been modernized and it is an excellent and mature package. It has everything you can need. It is very very good. It covers all the use cases you may have for inbuffer completion and it is highly customizable. What I'm showing you here is just some um interface to get you started. But at its core, you select uh one of these completions either by moving around or by typing further. When you are ready, you hit enter and you move on with your work.

[00:11:42]Okay, let me disable company and let me enable Corfu mode. So Corfu is a package that I believe was introduced in around the time of Emacs 28. So it entered the scene when the completion at point functions were the new normal. So it didn't have any legacy to speak of and was able to optimize for the completion at point functions. As such, Corfu is very fast, very efficient, very effective. It just works out of the box and at its core, it's this popup. Okay.

[00:12:19]And then you can configure it the way I have done here such that when you linger over one of those um candidates it will produce a popup if there is a pop-up with documentation. So if there is documentation you will see it there.

[00:12:35]Okay this is the idea. Again I don't want to be labor the point. I don't want to be exhaustive and enumerative to cover all the options. I want to remain generic here just so you see what is available, you get an idea and then of course you will check the relevant uh documentation and the relevant resources. Okay. So let me disable now corfu mode and now let me enable something that has been uh available that was introduced to Emacs. It's called sorry completion preview mode.

[00:13:07]Completion preview mode here. This was introduced in Emacs 30 and unlike the other two is meant to be super minimalist or the other three if you want like the default completions company and call. So here if I type something for example de a completion preview mode will provide a gentle suggestion one at a time like this and it's some shadowed text uh which uh it's telling me like it could be this maybe that you want but if I want this of course I can do tab sorry I can do uh I could do tab yes to complete it or like that sorry I have a a setting here which I um disabled. But the idea is I can select it or I can continue typing.

[00:13:54]Okay. So I can select it. Maybe I have it with enter. No. Yeah, I disabled it right before the video. Sorry, I messed up. But the idea is that you type and then you can either accept this suggestion or you can cycle through the suggestions. Here is the idea. You can cycle through the suggestions. If you don't like a suggestion, you just continue typing and things happen like this. And when it's a unique suggestion, well, you can just more quickly expand it like that. Again, I don't want to cover everything. The point is that you can use this if you like a more minimalist presentation, you can have this. Note though that uh company mode, if you are using company mode already, it has the option to do the equivalent of this, meaning to uh provide some shadowed text in front of what you are typing. So depending on what you have, you may want to stick with company or you may be like, okay, I don't need company. I can just use the built-in completion preview mode. Okay. Now, h a few more words about the back ends. So I said that the completion at point functions are fundamentally what we want to be using. Okay, it's the core. It's the essence of how things work here.

[00:15:09]Okay, the main providers for completion at point functions are major modes, specialized minor modes, the language server protocol and then packages that are designed to uh give you completion at point functions. One such package is cape. So let me just now switch uh to uh my org file here. And here I have a sample configuration based on my preference. It's for your eyes only.

[00:15:40]It's not meant to be exhausted. But here I have a sample configuration for cape.

[00:15:46]The idea is that cape gives you some backends. Okay. So you can either use the cape backends exclusively or you can combine them with other backends such as what LSP provides, what a major mode provides, what um um maybe some other provider provides. Okay, but the point is that you can have cape set up and the idea uh here is that you can have a global value and then you can also have buffer local values. One way to define buffer local values is to do a set q local and have a function that calls set q local and that function is bound to a hook. uh when you bind it to a hook such as this prog mode hook it means that it will affect progue mode and all the derivative modes basically all the programming modes same idea for text mode it will affect all the derivative modes of text mode and that includes markdown that includes org mode okay I don't want to go into the technicalities of the code I have written here h not least because you could do it a different way I don't mean to say that this is the way what I do want to focus on however or just at least mention is um a concept in completion at point functions which pertains to exclusivity of results. So I said earlier that completion at point functions is a list of functions and DMAX will go through the list and call each function one at a time until one returns results. If it returns results it stops there. Okay. So if you have one function here and one function there and this function returns results you don't see the results of the other function and depending on what you are doing you may want to see the results of more than one function. Okay so you don't want to have exclusivity you want to have functions that group together that pull their uh results. Okay. And they group together the results and you see everything. In the context of cape, there is a macro that does this for you.

[00:17:53]Of course, you can do it without cape, but the idea is that cape makes it more convenient for you. And basically, it's this idea of what cape calls a super cap f super completion at point function.

[00:18:07]Okay. And the idea of the super capf is that it groups together many individual cap fs completion at point function. And I don't want to go into the technicalities of what I am doing here.

[00:18:19]You can uh check how to do this yourself. Cape also has documentation. I decided to do it this way. But fundamentally the idea is I want those three to be one group. And so when I am uh doing a completion uh I want you to um pull these and I want to see them together. But if these do not return something, I want the files to be separate. So I don't want suggestions for file paths to be bundled up together with emoji and stuff. Okay, they are irrelevant. So I only want to see relevant results together. Okay, this is the idea. Don't focus on what I am showing here in terms of code. Just understand the principle one function at a time or a list of functions which together form a group. Okay. So let's now go and see the equivalent for company. So cape is a separate package that provides completion at point functions. Corfu doesn't provide any completion at point functions. It just ships the front end. In the case of company also because of its legacy it comes with its own back end as well as the front end that I demonstrated um but of course company is designed to work with the completion at point functions nowadays I don't want to go into the technicalities again it doesn't really matter but here is the gist of it you will configure the company backends and to use the completion at point functions you want to have company capf as one of the company back end probably you want it to be the first and if you want to have the equivalent of a super capf of what cape calls a super capf you want to group your um backends like this okay so this means that they are now a group and they are thus not exclusive but you see the files here this is exclusive okay and again because this is a variable you can do the same thing you can employ the same tactic such that you have a function which will be called in a certain context and it will set a buffer local value and this is the idea I don't want to go into the technicalities okay it doesn't really matter what you use the point is that you might need to augment what Emac is using for completion at point functions so there is company there is cape and of course if you are using LSP uh LSP will provide its own completion at point functions.

[00:20:56]So maybe you don't need anything else actually. Uh just to say that the preview mode, the completion preview mode doesn't ship with any cap FS of its own. It doesn't provide any backends.

[00:21:09]It's strictly a front end. It is meant to be minimalist. So if you really want to use completion preview mode uh and want also to have some additional cap fs, you either want to combine it with cape or you want maybe to combine it with cape and company like the documentation for cape explains all that or you just use completion preview mode with LSP. Depending on what you are doing, you may uh want to configure things one way or the other. So this is the idea folks. In conclusion, I want to say that Corfu and Cape are developed by Daniel Mendler. Daniel is the developer of Vertico and Consult among many other packages. Excellent packages I must say.

[00:21:58]Uh company mode is developed by Dimitri Gut who is also a contributor and maintainer of uh various subsystems that are in core Emacs. again an important uh developer in the EMAC space. And finally, completion preview mode is developed by Shel Yaron who is also a contributor to core Emacs. Many useful pieces of functionality there as well.

[00:22:24]The completion at point functions have over the time enjoyed contributions from many developers. Uh I think chief among them is Stefan Moner, but there are of course many others. So thanks uh to everyone for their contributions. Uh they make Emacs a better editor for the rest of us. So that's all for today folks. Thank you very much for your attention. Take care and goodbye for now.