Emacs org-mode
Welcome to the Emacs org-mode category.
You can subscribe to this category only via RSS!
I am working on restoring some of my related posts from my old blog, so keep checking here for new content.
Lately, Emacs on my Mac has been crashing more frequently. I don’t know for sure what it is, and I can’t get much from the logs, but it seems to be somewhat related to images. Perhaps some memory issue.
My research has led me to suspect that perhaps emacs-plus from Homebrew would be better (I’ve been using Emacs for macOS), but I’m not quite sure yet.
Right now I’m too tired to tackle it, so I’m going to crawl to bed. I’ve been too tired and too busy in the evenings to do much of my personal projects, which sucks. Hope to get some stuff done this weekend, but it’s going to be a busy one.
Denote includes the option of adding different directories, so this now makes sense:
(setq denote-directory
(if (eq system-type 'gnu/linux) ; If I'm using Linux, include the private folder.
(list (expand-file-name "~/Sync/Notes")
(expand-file-name "~/Documents/private"))
(list (expand-file-name "~/Sync/Notes")))) ; If I'm using anything else (macOS) just Notes.
(setq denote-excluded-directories-regexp "data") ; the data folder (with attachments) is excluded.
I have a private folder for Denote notes on Linux only. I used to only sync my Sync/Notes folder, which includes my informational notes and blog posts, but that means I’m missing out on Denote’s abilities in my private folder, which is slowly increasing in size. This solves this problem.
There’s no need to list the data folder (which includes my attachments, mostly in file formats that are not Denote’s format), so that’s why the exclusion is there.
Org files to beatiful docx files with Pandoc
When it comes to sharing written documents with my co-workers, I usually need to use Word or PDFs. Org-mode includes an export option to odt files, which does the job in a pinch, but it’s a bit harder to customize, especially if I want to preserve certain formatting elements. In the past, I used to create ODT files, import them into Word, make whatever changes I needed, and save them in a docx file. The problem there, besides the manual work, is consistency. If I want to make sure I use the same style for fonts, header sizes, and table formats, things get annoying fast.
About a year ago, I discovered that Pandoc can use an external Word document (docx) as a style template. As long as I have this file accessible, every org file I convert to docx will automatically (and beautifully, I may add) retain the exact style changes I need. This, of course, can be automated later inside Emacs, so the export process is basically seamless.
The problem lies in the fact that you need to work with Word and understand how its styles. Readers of this blog know I love to complain about Microsoft’s UI and its lack of consistency, and the instructions here are one fine example in my opinion. Be it as it may, I will try to keep the snarky comments in check.
Creating the default style sheet
The first thing to do is get Pandoc to create its default style reference document as a .docx. This can be done with pandoc -o custom-reference.docx --print-default-data-file reference.docx.
What you’re telling Pandoc here is to output (-o) a file named custom-reference.docx (you can call it whatever you’d like, as long as it’s a docx, since you want to work with Word). What to output goes into this file? The default reference document.
Once you do that, you will have a custom-reference.docx in the folder you ran the command from.
Now it’s time to work inside Word.
Modifying the styles
What you will see is a rather plain file listing a bunch of styles and their name. First, you will see “Title,” and it will be nice and big because it’s using the Title style in Word. Heading 1 will use the Heading 1 style, Body Text (further down) will use the Body Text style, and so on.
What you want to do is to access these styles in Word. You will be modifying them to fit to your liking.
To do that, search for the Styles Pane. In my case, Office 365 for macOS, I can find it in the ribbon under Home, but I wouldn’t be surprised if that’s a little different if you’re using Windows, or a different version of Word.
With the Style Pane open, you will see a long list of the available styles. You can navigate this way, but I find it’s easier to just look for them in the document to the left and select them; it will then show under Current style: to the right.
Now that you have a style selected, select the name of that style from the Style Pane and choose Modify Style…1. You will see the options available to you, such as the font name, text color, alignment, etc.
Make sure that you do not change the Name: of the style! You’re modifying existing default styles that Pandoc recognizes; if you create a new one (by giving it a new name), Pandoc will not know what to do with it and will ignore it. Remember, you are modifying a reference sheet of existing styles, not creating new styles. Likewise, this means that you have no reason to change anything in the document itself (the content) - Pandoc doesn’t care about that and will ignore it. You’re only touching the options in the Modify Style… window.
That’s basically all there is to it. As you will see soon, some items can get a bit more complicated as they are not available in the same way. Save the files when you’re done modifying the styles. You will probably keep going back to this document as you export docx files, tweaking things until you like them.
Using Pandoc to export from org-mode to a docx file
The next step is to tell Pandoc to convert an existing org file to a docx file using the reference sheet we just worked on. To do that, use the following command:
pandoc -s <input_file.org> -o <output_file.docx> --reference-doc custom-reference.docx.
The idea is the same as before, but we use a stationary option, -s (the reason for that is that we’re telling Pandoc to use a “full” version of a document, needed with the reference doc option, otherwise it can’t use our reference document).
Then, we’re pointing Pandoc to our org file and using the output, -o, as the doc file we want, then finally using the reference-doc option by utilizing our reference file, custom-reference.docx (unless you change the name to something else). That’s it, Pandoc will create the file in the folder you are in.
Images and Tables
For me, things get a bit more complicated. My technical documents often include screenshots, and I need those embedded in the resulting Word docx file.
The above command will embed images in the Word document, as long as org-mode can find and display your images (using the inlineimages option in org-mode). My org-mode technical notes are all written with Denote these days, which keeps everything organized, including the images, which are inside the default data subfolder. If your workflow is different, be mindful of your path to the images; you’d probably want to use an absolute path in org-mode (use C-u C-u C-c C-l when linking an image instead of the usual C-u C-c C-l for this). There’s also the option to extract media in Pandoc (see the –extract-media option). In my opinion, it’s simpler and cleaner to dedicate a folder for your org documents, completed with a subfolder for attachments.
Another issue that stumbled me was the tables.
Pandoc’s default style for tables is minimal - without borders. This might look nice if you want to hide your tables, but if you need a table with borders and perhaps some shading, you’ll run into issues.
The problem is with Word. The style for the table is not included in the Style Pane. In order to change the style for your table in the reference doc, you need to locate Table Design in your ribbon. Notice that if you work on a small screen (like in my recording), you might have to expand your options to find it. Then, you will have to extend the pane further and locate the Modify Table Style below. Once again, this is true for my case, Office 365 on macOS. If you’re using a different Office version or on Windows, this menu might be located elsewhere.
Once you have Modify Table Style open, you can change the borders and shading to your heart’s content, but as before, make sure the style name remains Table. If you create a new style, Pandoc will ignore it.
A few tricks I picked up
Under the name of the style, in the Modify Style or Modify Table Style window, you will see the Style based on option. You can change this option and browse through different colors and fonts to find something you like, and then customize further instead of starting from scratch. As long as you don’t change the name of the style, you’re OK. I found this useful when working with tables, as I was looking for a table with shading on every other row.
Speaking of tables, the border button is the little square icon in the middle. I spent a few moments trying to find it.
Bonus: use dwim-shell-command for quick export
If you’re an Emacs user, you like shortcuts, and you like to use shell commands from within Emacs. If you haven’t yet, check out dwim-shell-command.
Here’s how I have it set up. All I have to do is mark the org file(s) I want to convert to a docx, and that’s it. It doesn’t matter if I want to export only one file or fifty; it’s the same ease of use.
(defun jtr/dwim-shell-command-pandoc-org-to-docx ()
"uses pandoc to convert an org file to docx using a template docx file. The docx template file is stored in my Notes sync folder"
(interactive)
(dwim-shell-command-on-marked-files
"converting from org to docx"
"pandoc -s '<<f>>' -o '<<fne>>.docx' --reference-doc ~/Sync/Notes/Info/custom-reference.docx"
:utils "pandoc"))
Footnotes
1 : Or, another way to work is to use to select the text you want to modify, then go to Format and select Style from there; you will need to then choose Modify in the window that shows.
A bit of a teaser:
This bit of dwim-shell-command magic works nicely:
(defun jtr/dwim-shell-command-pandoc-org-to-docx ()
"uses pandoc to convert an org file to docx using a template docx file. The docx template file is stored in my synced notes folder"
(interactive)
(dwim-shell-command-on-marked-files
"converting from org to docx"
"pandoc -s '<<f>>' -o '<<fne>>.docx' --reference-doc ~/Sync/Notes/custom-reference.docx"
:utils "pandoc"))
Why I use Denote?
When I mentioned Denote again a couple of days ago, Omar commented that whenever someone mentions using Denote or org-roam, he’s curious to know if that person tried vanilla org-mode first, without additional packages.
I get it. After all, I’ve been resisting the whole Zettelkasten bandwagon and org-roam with it for a while, keeping the same opinion about org-roam Omar seems to have about Denote (I’m speculating here, of course). In general, I don’t like the idea of Doom Emacs or Spacemacs for the same reasons. Denote, for some reason, never registered this way. I liked it from day one.
As I was answering his question and he asked for more details, I thought it might make a good post. So here we are.
Not a whole lot changed for me since I first looked into Denote. I still mostly use it for my blog posts (like the one you’re reading right now) and my info notes, which mostly contain technical how-tos to myself and to a lesser degree, some records and memos, like a list of equipment in my home or who’s who at work. At this point, I also have a third, personal folder with notes only on my Linux computer as well, which mostly contain “supplemental” notes as I call them to journal entries I want to expand on in private (I use my work iPhone with Journelly, which does a fantastic job, but I don’t trust Apple or any cloud service for that matter with my private stuff).
It has been slow progress, but I’ve improved my usability with Denote, which I recently enhanced further after having the pleasure of talking to Prot 1:1 in one of his online tutoring sessions (always a pleasure, and highly recommended if you ask me).
The main philosophy behind Denote is something I’ve been in agreement with for a long time, before I started using it, or Emacs, or even got familiar with Linux: the idea of how to date and rename files. I’ve never agreed with the American way of writing dates, and I always prefer the ISO format: yyyymmdd followed by hhmm. To me, this is how things should be organized. So when Denote came around with Prot explaining that this is how he sees files should be organized - date, followed by keywords, and then a title - it just clicked.
You can extend this file-naming scheme to all your personal files if you’d like (at least all the files you visit with Dired), and I’m in the process of doing exactly that. Because the renaming function is built into Denote, it’s easy to stay concise and avoid mistakes (for example, renaming one file 20260124 something, and then another 2026124, emitting the 0 for January). While it’s true you could easily do that without Denote, to me, this core piece was like a sign that said, “Hey, look at this, this guy is building a package that is based on something that makes sense, check it out.”
Tags are another important part of Denote. Org-mode has tags built into headers, which works, but the headers, to me, are a bit too specific to need them. For the most part, I can get the level of information I need through the header itself, and I use tags to associate a certain header with a person - but even that is not useful, as I often work with too many people for the tags to really mean anything. Meanwhile, it’s the files themselves that need tags: certain workflows are similar, but are tagged to work for different operating systems (say, how to set my work VPN on Linux vs on my Mac), or maybe I have a naming convention for different departments at work, and I want to tag the relevant departments.
I didn’t always break my different workflows and notes into files. In the past, I had one org file as a wiki: a single file with many headers explaining workflows and procedures. I often had errors there. It was one big file that I often opened and edited, so I had syncing issues, headers that got mixed up, etc. The other problem I had with the wiki was logical: at times, a certain subheader would fit under two different headers, and I would have a hard time deciding where to place something. Headers work as categories, while tags work as ,well, tags. If we take the VPN example from before, I could put my VPN instructions under “networking,” “security,” or “remote work,” but not all. I don’t have this problem with Denote.
Another, more recent skill I started using with Denote is its metadata and search features. Meta notes work as an index of other files, where I can have one file with a dynamic list (updated whenever the file is loaded) of live links to other files with the same tag or other regex I use. Meanwhile, searching with Denote using denote-grep (something I learned recently) shows a list of all the files matching my search in collapsible form, making it easy to find what I need. I can use one of many denote-query forms to filter those results even further, and of course, there’s denote-consult, which, well, if you know consult, you know what it does, and it’s excellent.
Finding stuff is a central issue for anyone keeping notes, no matter what system they use. Denote supplies me with the best tools I’ve seen for this job. It’s true that many of these tools are already available in Emacs and in org-mode. Denote doesn’t alter these existing options; rather, it builds on them. In fact, that’s a big part of Denote: the way its commands are constructed borrows from what Emacs already does, so if you’re familiar with one, you intuitively know what the same thing would do inside Denote.
Another example of the above is the org files you create with Denote. You can use Denote directly with a capture template, and each file is created with file tags that make sense if you use org-mode: #-title, #+date and #+filetags, which denote changes automatically as you rename files with it. These work together with additional options I include in my notes, such as #+STARTUP: inlineimages. In a Markdown export, such as this very post you’re reading now, Emacs knows to ignore these org-mode only options, so I don’t need to clean them out when I’m ready to post. It’s a nice touch that already exists in vanilla org-mode, but with the added benefits of Denote, like having my post tagged, dated, and easily found in a search alongside the rest of my Denote files in the future.
There are definitely more features of Denote I don’t use, or use enough (blacklinks, which I believe are more of a recent introduction, is one of them), but as I stated above, this is a process. I learn more every week, and as my notes collection increases, so does my understanding of how to use Denote in a way that works for me.
Spent the last day and a half on and off figuring out how to use pandoc to make my org-mode files in pretty, readable docx files, including tables. It was a challenge, but it’s worth it. The key is to use a reference doc and to know how to use it. Need to expend on that.
Reflection on my Emacs experience
A couple of days ago I found out org-mode capture sun menus can handle additional nested menus. To recap quickly here (see more details in the link), this means that you’d have the main menu for capture templates, which will lead to additional templates menus, and that menu can keep leading to more and more menus. I am not sure how many levels of these templates we can have, but “more than enough” is a good enough answer for me.
As I was writing the post above, I went back to my old blog and reviewed two of my older, Emacs-usage-defining posts, and added them to this blog: Org-mode in files and Submenus in org-mode capture.
Those of you who enjoy reading about the Emacs experience from a user perspective (especially someone who started out without a programming background) might want to give those a read, in the order mentioned above. Beyond explaining these crucial org-mode methods (and in my opinion, crucial to use Emacs in general), these describe the struggle and the learning process of, well, how to Emacs correctly: from realizing an option exists (of course it exists, it’s Emacs), through asking the right questions, to finding the information and implementing the solution.
I’ve spent a couple of hours re-reading and revising these posts, and reached a conclusion that many of you who are reading this over their RSS feed in Emacs would probably agree with. We, Emacs users who blog, have a critical role in the Emacs ecosystem: we provide others with ideas and questions to ask, which can later be directed to official channels, the manual, and brainstorming answers.
As I mentioned a couple of times throughout the lifetime of my blog (this one and the old one): Emacs is not just a program, it’s a lifestyle.
org-mode capture: menu inside a menu
Over the weekend, I modified my org-capture templates again. I was curious to see if I could create a second-level menu in org-mode capture under my already existing menu items - and as it turns out, it’s possible, and also pretty straightforward. Here’s the start of the setting (keep in mind it’s not complete, don’t just copy-paste this):
(setq org-capture-templates
(quote (
("w" "work")
("wt" "work-ticket")
("wta" "work-ticket department A" entry
(file "~/Sync/Work/department A.org") (file "~/Sync/Templates/temp-ticket.org"):prepend t)
This will result in the following:
Select a capture template
===========================
[w]... work...
then:
Select a capture template
===========================
[wt]... work ticket...
then:
Select a capture template
===========================
[wta] work-ticket department A
I should explain this further, probably first explaining why I need such a complicated system of menus within menus, but every time I sit down to explain I run out of time. I wanted to put it out there first, and I’ll come around to explain it soon, hopefully.
I like the DU command in Dired
I had fun using the du command in Emacs this morning (that’s what I do when I don’t sleep well; judge away).
In Linux (and macOS), the DU command (I believe it stands for Disk Usage) is usually used to figure out what folders take up space on your hard drive.
While different GUI tools exist, they are not useful on encrypted drives on Linux, as they show the encrypted blocks instead of the directories, which don’t really tell you much (you could probably run those with escalated permissions, but I didn’t try).
For a gamer like me, it’s useful to see which games take up the most space. Here’s an example with a few useful flags, in the command line:
du -hs ~/.steam/steam/steamapps/common/* | sort -h
Shows me the following:
4.0K /home/jtr/.steam/steam/steamapps/common/Steam.dll
76K /home/jtr/.steam/steam/steamapps/common/Steam Controller Configs
100K /home/jtr/.steam/steam/steamapps/common/SteamLinuxRuntime
248M /home/jtr/.steam/steam/steamapps/common/Steamworks Shared
657M /home/jtr/.steam/steam/steamapps/common/SteamLinuxRuntime_soldier
776M /home/jtr/.steam/steam/steamapps/common/SteamLinuxRuntime_sniper
1.4G /home/jtr/.steam/steam/steamapps/common/Proton - Experimental
3.4G /home/jtr/.steam/steam/steamapps/common/Deep Rock Survivor
3.5G /home/jtr/.steam/steam/steamapps/common/Deep Rock Galactic
11G /home/jtr/.steam/steam/steamapps/common/Hades II
21G /home/jtr/.steam/steam/steamapps/common/Yakuza Kiwami
82G /home/jtr/.steam/steam/steamapps/common/Space Marine 2
132G /home/jtr/.steam/steam/steamapps/common/Helldivers 2
Some of the Steam-related components are not games, but… Helldivers 2 takes how much space?? Anyway.
A quick review of the options I used:
-hs for human readable (shows space in G for gigabyte and M for megabyte instead of writing in kilobytes) and summary (shows the top directory only)
~/.steam/steam/steamapps/common/* for the target path (this is where Steam stores the games in Linux, at least in Debian distros)
| to pipe the du command it into another command to read it better, in this case:
sort -h the sort command, which will sort it nicely again by order in human format. We need this last part if we want to see the directory in order, with the biggest one at the bottom.
Some places recommend using sort -hr, the additional r for reverse, which means in this case we will see the biggest directory at the top of the list. I don’t need it, because I want to see the biggest folder at the bottom, near the command line, which is where I’m going to focus next.
In Emacs, this command is easier use and works better thanks to Dired. Find a folder, mark it (m) in dired, and run dired-do-shell-command (!) and follow up with du -hs.
But since we’re in Emacs, and we might want to work with the results as text, we could use dired-do-async-shell-command (&). This will place the output in a temporary buffer we can work with (so we can save it to a text file, for example, with C-x C-w).
And here’s another thing I didn’t know: you can run these commands in Dired on multiple directories. Just mark several directories in Dired, and the resulting buffer will give you a list of all the directories you’ve marked. If you have this saved as a text buffer, it’s pretty easy to work withthe results (for example, save it as an org file and add headers telling you what you want to do with each directory).
By the way, even though it’s somewhat redundant with Dired’s default listing of files, you can also add the a option for du in this case ( for all) to display the files in the directories you’re viewing. This is useful in cases like the above, where you’re already working with the du command in Emacs and interested in looking at individual files as well, not just directories. Of course, you can just go in and list the files in Dired and open another Dired buffer with another directory listing files by size… this is Emacs, you have many ways to do whatever you want.
I think I’m going back to clocking in and out of sub-tasks, even though I said I wouldn’t. It’s easier not to, but the end result, I think, is messier, and I can’t get the clock reports right… on the other hand, it’s kind of an overkill. 🤔