Context:
I’ve used Emacs off and on for many years at this point. Throughout that time there have been periods where I really leaned in to it and tried to use it for everything, and there have been periods where I only used it for org and/or magit, etc. I’ve learned lots of things about it and I’ve forgotten lots of things about it, but I’ve never been what I would call an “expert” or even a “power user”. So, when I feel like something isn’t working well in Emacs, I almost always default to the assumption that I’m doing something wrong or misunderstanding something, etc.
So, it very well may be that I’m wrong/crazy in my recent conclusion that use-package might not be the ideal abstraction for managing Emacs packages.
With that out of the way, I’ll say that when I first saw use-package, I thought it was amazing. But, in the years that I’ve been using use-package, I never feel like my init file is “right”. Now, I’m starting to think that maybe it’s use-package that’s wrong and not me (insert Simpsons principal Skinner meme).
I don’t know how best to articulate what I mean by use-package being a “wrong abstraction”, but I’ll try by listing some examples and thoughts.
Autoloads
First of all, I feel like the way autoloads are handled with use-package is too mistake-prone. Libraries/packages typically define their own autoloads, but the use-package default is to eagerly load the package. I understand that installing a library via package.el, etc will process the autoloads for us and that manually/locally installed packages get no such benefit.
But, if we’re using use-package to also manage installing the packages for us (:ensure t
), then why shouldn’t it know about the autoloads already and automagically imply a :defer t
by default?
So, by default, we have to remember to either add :defer t
or we have to remember that setting our own hooks, bindings, or commands will create autoloads for us.
I know that you can configure use-package to behave as though :defer t
is set by default, but that’s just broken for packages that don’t have any autoloads.
It feels like maybe use-package is doing too many things. Maybe it was actually more correct in the old days to separate the installation, configuration, and actual loading of packages, rather than trying to do all three in one API.
Configuration that depends on multiple packages is ugly/inconsistent
Many packages are fairly standalone, so you can just do,
(use-package foo
:defer t
:config
(setq foo-variable t))
and it’s clean and beautiful. But, sometimes we have configuration that is across multiple packages. A real-world example for me is magit and project.el. Magit actually provides project.el integration, wherein it adds magit commands to the project-switch-commands and the project-prefix-map. That’s great, but it will only run if/when the magit package is loaded.
So, my first guess at using use-package with magit was this,
(use-package magit
:ensure t
:defer t
:config
(setq magit-display-buffer-function #'magit-display-buffer-same-window-except-diff-v1))
which seems reasonable since I know that magit defines its own autoloads. However, I was confused when I’d be using Emacs and the project.el switch choices showed a magit option sometimes.
I eventually realized what was going on and realized that the solution was to immediately load magit,
(use-package magit
:ensure t
:config
(setq magit-display-buffer-function #'magit-display-buffer-same-window-except-diff-v1))
but that kind of sucks because there’s no reason to load magit before I actually want to use it for anything. So, what we can do instead is to implement the project.el integration ourselves. It’s really just two commands:
(define-key project-prefix-map "m" #'magit-project-status)
(add-to-list 'project-switch-commands '(magit-project-status "Magit") t)
But, the question is: Where do we put these, and when should they be evaluated? I think that just referring to a function symbol doesn’t trigger autoloading, so I believe these configurations should happen after project.el is loaded, and that it doesn’t matter if magit is loaded or not yet.
Since, project.el is built-in to Emacs, it’s probably most reasonable to do that config in the magit use-package form, but what if project.el were another third-party package that had its own use-package form? Would we add the config in the project use-package form, or in the magit use-package form? Or, we could do something clever/hacky,
(use-package emacs
:after project
:requires magit
:config
(define-key project-prefix-map "m" #'magit-project-status)
(add-to-list 'project-switch-commands '(magit-project-status "Magit") t))
But, if we do this a lot, then it feels like our init.el is getting just as disorganized as it was before use-package.
Conclusion
This is too rambly already. I think the point is that I’m becoming less convinced that installing/updating packages, loading them, and configuring them at the same time is the right way to think about it.
Obviously, if you know what you’re doing, you can use use-package to great success. But, I think my contention is that I’ve been familiar with Emacs for a long time, I’m a professional software developer, and I still make mistakes when editing my init file. Either I’m a little dim or the tooling here is hard to use correctly.
Am I the only one?
But, if we’re using use-package to also manage installing the packages for us (:ensure t), then why shouldn’t it know about the autoloads already and automagically imply a :defer t by default?
Package installation and activation are two seperate concerns.
:ensure
ensures the package is isntalled if it isn’t already. Users might want to ensure a package is installed and prefer it to be immediately required.So, by default, we have to remember to either add :defer t or we have to remember that setting our own hooks, bindings, or commands will create autoloads for us.
I feel like you’re misunderstanding what autoloads do. Hooks do not autoload anything. The
:commands
use-package keyword does autolaod commands. This is useful when a package author has not provided the autoload cookie in the package for a command, or when you wish to forgo loading all of the autoloads.I know that you can configure use-package to behave as though :defer t is set by default, but that’s just broken for packages that don’t have any autoloads.
How is it broken? There are other ways to load a package. Namely,
require
.It feels like maybe use-package is doing too many things.
It only does what you tell it to do via user options and declarations.`
but that kind of sucks because there’s no reason to load magit before I actually want to use it for anything. So, what we can do instead is to implement the project.el integration ourselves
Or
(use-package project :ensure nil :defer t :config (require magit))
There are multiple ways to set this sort of thing up and use-package (which should have been nameduse-feature
) can be used to configure built-in features.Either I’m a little dim or the tooling here is hard to use correctly.
Third option: You haven’t taken the time to digest the use-package manual and/or expand the macro to see what it’s doing. It’s a DSL. You have to learn it to use it effectively.
Am I the only one?
You must be. Otherwise someone would’ve written “use-package alternatives”, which has an almost searchable ring to it.
Ultimately, organization comes down to the user. Tools like use-package make it easier, but they do not guarantee it.
(imo) you’re using it wrong, I made everything defer by default then rely on autoloads and/or :bind/:commands with light use of :after . 99% of things are lazy loaded after an hour or so of work on a > 1000 line .emacs . It’s not ideal and I should probably turn most of :commands into patches upstream for auto loads but I am lazy.
Can you give me an example where you use
:after
? Isn’t it pretty tricky to use:after
when also settinguse-package-always-defer
?
With that out of the way…
Your life story wasn’t in the way until you put it there.
Your question could be condensed to “use-package is extra bureaucracy when you need to do something non-trivial, e.g., reconciling two packages’ config.”
I found use-package impenetrable as a noob. It only started making sense after I learned the subtleties of minor modes, autoloads, keymaps, custom variables, etc., at which point use-package became superfluous macro alchemy. Judicious use of `eval-after-load’ is all you need and a lot less wtf.
That’s silly - use-package does two things:
- Download things
- require things
Your life story wasn’t in the way until you put it there.
Agreed here.
Judicious use of `eval-after-load’ is all you need and a lot less wtf.
Until you realize that
eval-after-load
is just a superfluous abstraction itself (a forgivable “noob” move) and transcend to manually managing load history and friends. /sYour life story wasn’t in the way until you put it there.
I find that adding my “credentials” or background sometimes helps when posting questions or discussions on Reddit. If I had posted that I found use-package’s handling of autoloading a little confusing without any context, I’m fairly sure that I would get a bunch of replies explaining how it works, rather than a discussion of whether the way it works is good.
But, basically, I agree with you that
eval-after-load
is about half of what I wanted from use-package–the other half being a way to declare which installable packages I want available.
People seem to be missing the point that no conceivable package loading tool — use-package, general, straight, by-hand gnarly lisp using eval-after-load, etc., etc. — will solve the original problem as stated. You want a binding to appear, tying project to magit. That binding is specified at the top level in one of magit’s many lisp files. You want this binding to appear without loading magit. There is no approach which will accomplish this, other than excerpting that binding yourself, as you’ve done in the “clever/hacky” solution (though I’d put it in the project :config stanza for better organization).
The way simple extension loading works in other applications in my experience is “fully load all configured extensions at startup”. This solves many problem (including this one), but is slow. You have the option to defer loading and much more with Emacs; the price you pay is complexity and the need to understand how the pieces fit together.
Maybe if Magit contained the following form?
;;;###autoload (autoload #'project-switch-project "magit")
i.e. autoloading an autoload :)
People seem to be missing the point that no conceivable package loading tool — use-package, general, straight, by-hand gnarly lisp using eval-after-load, etc., etc. — will solve the original problem as stated. You want a binding to appear, tying project to magit. That binding is specified at the top level in one of magit’s many lisp files. You want this binding to appear without loading magit. There is no approach which will accomplish this, other than excerpting that binding yourself, as you’ve done in the “clever/hacky” solution
Yes, agreed. My contention is not with the inherent complexity of the situation. Rather, my issue is related to this last part of your paragraph,
though I’d put it in the project :config stanza for better organization
That’s the whole problem with using use-package forms for configuration! use-package is centered around a single feature at a time, whereas many configurations involve tying multiple features together. In this case, it doesn’t matter if I put the configuration in project’s use-package form or in magit’s use-package form–in either case, one of the forms will be misleading because it will seem like all of the configurations related to that package is in that form, but it won’t be true because something about it is being configured in the other package’s form.
That’s not the only thing, either. Another example is completions. I use vertico and corfu, but they tie in closely with a lot of built-in Emacs completion stuff, so I have the following settings,
(customize-set-variable 'read-extended-command-predicate #'command-completion-default-include-p) (setq completion-category-overrides '((file (styles partial-completion)))) (setq completion-styles '(flex basic))
those settings are not part of the vertico package, but they’re being set because I’m using vertico and want it to behave a certain way. So, do they go in the vertico use-package, or do they go in an emacs or simple.el use-package, or do they just go at the top level somewhere?
The truth is that we don’t actually configure packages in isolation, which is why I feel like use-package kind of imposes a structure that appears to make sense in a lot of cases, but does NOT actually make sense in the general case.
That’s the whole problem with using use-package forms for configuration! use-package is centered around a single feature at a time, whereas many configurations involve tying multiple features together.
What would you propose as a better design? It seems like you hope that pretty much all users of such a mythical package system would converge on precisely the same init structure, given the same feature requirements. Given the complexity of the system, I don’t see how that’s possible. I mean not even for simple toml/yaml configs is that ever the case.
In this case, it doesn’t matter if I put the configuration in project’s use-package form or in magit’s use-package form–in either case, one of the forms will be misleading because it will seem like all of the configurations related to that package is in that form, but it won’t be true because something about it is being configured in the other package’s form.
You have two packages, and you’d like them to be tied together in some manner. Where should this config go so as to avoid such confusion? It has to go somewhere. The cleanest approach I know is to create a separate small package whose sole purpose is to make that integration happen, and use-package it. Some of these “tie-in” packages in fact do exist, for example:
;;;;; embark-consult: Tie the two together (use-package embark-consult ...
You could create your tie-ins as small user packages, and group them under a joint heading in your init. This is not as hard as it sounds: at base a package is just some .el file on the
load-path
which does(provide 'something)
. But even still, this requires you or a user of your init to notice this “combiner” package, and realize they need it if they want the joint functionality.So, do they go in the vertico use-package, or do they go in an emacs or simple.el use-package, or do they just go at the top level somewhere?
This is kind of like asking whether socks go in the top drawer or the bottom :). Wherever you’d best remember them! Vertico makes the most sense to me. I
use-package
builtins all the time, so as to organize them in a way I will remember. I do all my completion category stuff in myorderless
stanza, and for any “special” completion setups (e.g.eglot
), do those in the relevant package stanzas.M-x find-library
will show you all the things you canuse-package
.
One of the things I don’t like about it is the
:custom
part, if I want to change a variable I’ll have to re-evaluate the whole use-package form.That’s also true of the :map and :hook forms, no? They all use magical syntax sugar that is meaningless outside of the macro. Luckily, :init and :config basically just wrap their contents in a (progn) with some error handling, etc.
You are absolutely right. I use use-package less and less these days.
As for :init, whatever you put in there you can just put outside of use-packge right? And :config, I’m happier using with-eval-after-load.As for :init, whatever you put in there you can just put outside of use-packge right?
99% of the time, yes. As far as I know there are three “lifecycle hooks”: prelude, init, config. Prelude is evaluated when the use-package form is encountered no matter what (even if it’s :disabled, etc); then init is evaluated if the package is not disabled via :requires, :disabled, :if, etc; then config is essentially the same as
with-eval-after-load
.
> Maybe it was actually more correct in the old days to separate the installation, configuration, and actual loading of packages, rather than trying to do all three in one API.
IMO, yes, this separation of concerns is a good ideia – or at least the ‘installation’ phase from the others.
In my config, I use `package-selected-packages` and `(package-install-selected-packages)` rather than depending on `use-package` installation facilities. Code.
You are right, and I never use use-package
I don’t use
use-package
, but I’ve seen a lot of questions from users who do use it but don’t understand how to use it, or what it’s going to expand to, or what the things that it expands to actually do. My conclusion has been that for some users it introduces as many problems as it solves. I think those users would be better off if they learned how to manage their config without it first, and only considereduse-package
after understanding the more fundamental building blocks upon which it is built.It’s certainly not something you need to use, in any case. It’s clearly an invaluable system to many users, but if you don’t get along with it, don’t use it.
My conclusion has been that for some users it creates more problems than it solves, as they can wind up with a new layer of things they don’t understand, and one which further obfuscates the systems they didn’t understand to start with.
There are many questions about use-package because new users are encouraged to copy/paste configurations from package READMEs and other configurations. If you take use-package out of the equation, you’ll be left with questions about the underlying elisp. It doesn’t matter what was used if the crux of the question is “I copied this thing I didn’t read about and now I don’t know how it works. Explain it to me?”
I think your problems are due to not understanding how Emacs works under the hood. To me, the only problem with use-package is that it hides those details and makes people learn a DSL that use-package is, instead of learning simpler mechanisms of with-eval-after-load and hooks which are used to implement the functionality of use-package under the hood. In past years I have seen many questions here on Reddit and on SX related to use-package because people don’t understand Emacs under the hood. But that is about it; I see no other drawbacks than that.
Personally, I have stopped using use-package after about 2 ~ 3 years ago. Been using it for a few years before that. The reason why I abandoned it was actually because I wanted to learn how such a thing was implemented and how Emacs worked under the hood. Another reason was that use-package was not included in Emacs back in time. However, I am not going back to use-package, just because it got included. Partially I am too lazy to rework my setup again, and partially I personally don’t need it.
To be clear, I don’t agree that use-package is wrong about anything or poorly written or anything like that. On the contrary, use-package has its very good points. One of them is that everything related to a single package is condensed into one place. At least configurations that are built around the use-package tend to be well structured. That part itself is a very good reason to use use-package, unless you have the very good discipline or come up with a system to organize your init file.
I think your problems are due to not understanding how Emacs works under the hood. To me, the only problem with use-package is that it hides those details and makes people learn a DSL that use-package is, instead of learning simpler mechanisms of with-eval-after-load and hooks which are used to implement the functionality of use-package under the hood. In past years I have seen many questions here on Reddit and on SX related to use-package because people don’t understand Emacs under the hood. But that is about it; I see no other drawbacks than that.
As a software developer, if you have to understand (exactly) what is happening under the hood of an abstraction in order to use it correctly, then it’s not a good abstraction. The entire point of abstraction is to allow us to think on a higher level, and I would say that use-package fails that test because it has a ton of nuance and subtle self-interactions that requires me to more-or-less just macro-expand all of my declarations to see what they are actually doing.
The fact that so many people struggle with it gives credence to this as well. Frankly, Emacs is complex and complicated. It has lots of legacy baggage and idiosyncrasies, so I would expect people to get their
with-eval-after-load
s wrong or their keybinding syntaxes wrong. But, use-package seems to hurt as much as it helps.if you have to understand (exactly) what is happening under the hood of an abstraction in order to use it correctly, then it’s not a good abstraction.
Nobody says you have to understand it exactly, but you have to have an understanding of what is going on. You also have to have an understanding of the abstraction itself, of what are you using. Have you even read the use-package documentation? I don’t recall ever that use-package was meant to hide away the Emacs itself, but to help you write more structured setup. If you don’t like it don’t use it, it is not harder than that. I don’t use it myself. But your problem is neither Emacs nor use-package. I personally can’t care less if you use or not, but you are now blaming the missunderstanding on external factor. Lots of people are using use-package and find it useful. Perhaps you should reflect over if it is the software or the user :-). I don’t mean to be impolite or arrogant, but sometimes things feel difficult when we are not ready for them. Nobody promised that use-package should be trivial, albeit I personally don’t think it is very difficult tbh.
You have two packages: X and Y. You want to do something with Y if X is loaded, but sometimes you want to just “jump” into Y without loading X, and than wonder why your X based setup will not load. In Emacs your vanilla option is to always load X whey you load Y, or cook your own thing. In your particular example, your option is to load project.el whenever you load Magit, it is not harder than so. Just require package.el in your Magit configuration.
requires me to more-or-less just macro-expand all of my declarations to see what they are actually doing.
Welcome to Lisp :-). You can compare that to C++ where you more or less have to look at assembly output if you care about efficiency and what your compiler does behind your back. I think you should actually be thankful you can do that. Try to do that with Python, JS or you name it.
The fact that so many people struggle with it gives credence to this as well.
I would rather care how many people do not struggle when looking at the big picture of how many people actually use use-package.
Frankly, Emacs is complex and complicated.
Sure, but so is any piece of software that does non-trivial tasks. If you find Emacs too complex and complicated, to the limit that it causes too much frustration and time loss, don’t use it, nobody is holding your hands.
It has lots of legacy baggage and idiosyncrasies
Sure, I agree. I personally use to say that Emacs is a hack over a hack. And I am really surprised how well it runs and does what it does, how hacked together the source is. But is to expect of 50 year old software. It is developed by many people with different backgrounds and different goals, mostly by hackers who just wanted to bend Emacs to do their own thing. It is a hackable editor made for hackers.
I would expect people to get their with-eval-after-loads wrong or their keybinding syntaxes wrong
I wouldn’t. Mostly because those two particular things are incredibly simple to get correct.
use-package seems to hurt as much as it helps
I think it is individual. What you did wrong in your example is that you haven’t required project.el in your Magit configuration. Do it and your problems will be gone. Simple. Nothing wrong with use-package.
I can also tell that you can just use with-eval-after-load and mode hooks to achieve the same lazy setup if you prefer not to use use-package. I don’t think it is that hard; I have done it myself and I was not even an experienced Emacs user or Emacs Lisp programmer at that time.
I can understand your frustration; I have been there myself, and my best tip is: start reading the documentation and built-in help. It is a best first step towards understanding Emacs or user-package one can take.
I have to correct you: their example is more about making sure Magit will be loaded so it configures
project-switch-commands
, before they callproject-switch-project
. And don’t “solve” it by just loading Magit on init. I suppose naively it could be solved with(autoload #'project-switch-project "magit")
and I’m not sure what’s the equivalent use-package expression. You have to be careful not to lose the usual guarantees of use-package wrt. graceful degradation (we don’t want an error “unknown file magit” – which is the whole point of use-package, otherwise you may as well just write the above form).
General is a package that really illuminates the tension that occurs when the user wants to integrate two packages.
:after
is okay for a quick solution, but for packages like general, you would end up with one ensured package implicitly depending on everything.It’s much less bad that you end up loading everything and much worse that the use-package declaration feels anything but modular or independent, instead incorporating symbols from all over Emacs and implicitly depending on other use-package hunks.
What is still a much deeper problem in my opinion is the lack of being able to superimpose and compose configuration. This is the reason why adopting random hunks of use-package declarations is not safe and why almost no two users can easily re-use large units of configuration.
You have the wrong target. I don’t think you are (or should be) actually complaining about use-package, but about the setup of various cross-package “tie-in features”. So magit knows about project.el. It kindly offers to add a special command to
project-switch-commands
, but (quite reasonably) only after project.el is loaded, and ifproject-prefix-map
is defined. Luckily the latter is a builtin autoload, so it’s just a matter of ensuring project is required with magit before you can use the tie-in, whenever you want that to happen.As already pointed out, you could add
(require 'project)
directly in magit’s use-package:config
stanza to make this tie-in happen right away after magit loads. Or, perhaps even better, identify a hook (magit has many) to do this “just in time” before you use magit in some meaningful way. If you want this tie-in to happen somehow magically without loading magit, good luck with that; it’s a literal top-level(with-eval-after-load 'project)
in magit-extras.el! You only option is therefore to do what you did: boost that logic out into your config. Or how about this:(require 'magit-extras)
in your(use-package project :config)
. That way you’re only loading a tiny part of magit when project.el is loaded.The bottom line: how is
use-package
or any other config paradigm supposed to know when you’d like magit’s project.el tie-in to activate? Some people would like to have project -> magit interaction on demand the very first time they invoke the project menu in a git-controlled file. Others want that tie-in command available only if they have already been working with magit in a session. You apparently want it active and ready to go without loading either package. This is way beyond use-package’s scope. The only solution is to investigate the nature of the tie-in, and configure it yourself to work the way you want. Which is what you did ;).Wrong or not, it seems it’s completely unnecessary in Emacs 29 for me.
.emacs is so much cleaner now.Since, project.el is built-in to Emacs, it’s probably most reasonable to do that config in the magit use-package form
A key tip which took me a while to internalize: you can create use-package stanzas for built-ins. I do so all the time. E.g.:
;;;;; emacs server (use-package server :config (unless (server-running-p) (message "Starting Emacs server...") (setq server-client-instructions nil) (server-start)))
I have a tangential observation to contribute. It seems like `use-package` is the way to configure settings, yet customize-group, customize-variable make no use of it. There is a mismatch, others see that too?
I think you are right, at least for problems of configuring multiple packages together. I think there may be another useful way to organize configs, but I haven’t come up with anything yet. For now use-package is pretty optimal.