Plain-Text Writing Lesson Six: Citation Wizardry

Note: if you’re new to this series or to plain-text writing in general, please refer to¬†Lesson One of this series to get started.

libraryAs I mentioned in the previous lesson, one of the biggest pains for me in academic writing is dealing with what happens to footnotes and citations in Word documents over the course of my writing process. Every piece I write goes through many stages of life: it starts as random jottings, becomes a general “idea draft,” then becomes a conference paper or a blog post, then an article or chapter, then a presentation, etc. Each of these versions goes through several revisions and I don’t know how much editing and refining. This means that citations get dropped, added and moved around; whole sections of one version get cut and pasted into another, sections get moved around, etc. And, of course, as I adapt a certain course of research to various publications and audiences, the style and formatting of the citations changes: one venue wants Chicago style footnotes, one wants endnotes, one wants MLA, ad nauseam.

One of the main issues that drew me to a plain-text workflow was what kept happening to my footnotes in word processors (Word and LibreOffice, for the most part), as my writing went through all these convolutions: it seemed that every time I moved things around or changed formatting, something would get messed up. When I converted from footnotes to endnotes or back again, the process never seemed to work as advertised, and I’d spend hours cleaning up formatting and manually fixing citations, even when using a citation manager like EndNote or Zotero (and sometimes because of the ways in which auto-inserted citations can get screwy when edited manually). If I needed to change style from MLA or Chicago or back, I’d wind up needing to go through an entire document and redo all the notes from scratch. All of this is not to mention the “cascade” effect, which I suspect you’ve experienced, of changing something in one note only to somehow throw off the formatting of the rest of them, or having note formatting fail to behave as expected, or even having notes change position or show up in the wrong place. There’s nothing more frustrating than pushing a deadline for an article only to find one’s self wrangling unruly citations.

There’s a better way.

Enter Pandoc once again, with some special extensions.

As with other elements of markdown writing, the idea here is that you insert citations as plain text right in line with the rest of your plain-text document. This means, for one thing, that citations always stay where they’re supposed to, even when you’re moving blocks of text around. The actual formatting of citations into different documentation and note styles (in-text citations, footnotes, endnotes, MLA, Chicago, APA, etc.) is something that only happens after your document is finished and ready to be prepared for publication. Just as you can transform any markdown document, via pandoc, into any digital format (Word, PDF, html, etc.), you can convert the plain-text, inline citations in any markdown document into any desired documentation style, without changing the original text. If you output to Word format at one point and then, say, make major edits and move sections around in your original plain-text document, you simply convert the plain-text document again and everything stays where it should and looks like it should with very little, if any, fuss.

Making this work might seem a little bit complicated at first, but I’ll try to detangle the process for you here. It’s really quite quick and efficient once you get the concept.

One thing I’ll assume in this post is that you’re already familiar with or using a citation manager such as Endnote, Zotero, or Refworks (I prefer Zotero). If you’re not, you’re really missing out on a handy tool, so I’d certainly recommend doing a little research and installing a citation manager that works for you. Zotero is clean, effective, versatile, as well as free and open-source, which is why it’s my manager of choice. In what follows, I’ll demonstrate the process with instructions for Zotero where needed, but everything I do with Zotero should be easily adaptable to the citation manager of your choice.

Step One: Enhance the Magic Wand:

To make citations work with pandoc, you need one additional tool, an extension to pandoc called pandoc-citeproc.

In any Debian-based version of linux, you can install this simply be opening a terminal and typing:

sudo apt-get install pandoc-citeproc

If you already installed pandoc in Windows, you should already have the pandoc-citeproc package, since it’s included in the Windows MSI installer, which you can find here.

Step Two: Gather the Elements:

In order to fully process citations in markdown, you need three elements:

  1. A bibliography file in bibtex format that will live in the same directory as your main markdown document. This is the file from which pandoc will draw the actual bibliographic data.
  2. Your markdown document itself, with correctly formatted citation placeholders (concerning which more below).
  3. A .csl file containing the formatting information for whatever documentation style you need or prefer.

Let’s set up these elements one by one:

A Bibliography File in Bibtex Format

This is quite easy to do. Assuming you’ve been keeping your bibliographic data in your citation manager, all citation managers have the ability to export a collection of citations into Bibtex format. In the case of Zotero, the procedure is:

  1. Open your Zotero app
  2. Right-click on the collection you want to export and select “Export Collection”
  3. In the ensuing dialog box, select “Bibtex” as your format, and click OK
  4. Specify a location for your bibliography file–this should be the same folder in which your draft document resides.
  5. Name the file whatever you like, and make sure it has a “.bib” extension.

This file will simply live alongside your text file in the same folder, so the citation information it needs is always handy. If, in the source of revising, you wind up adding sources to the original collection in your citation manager, simply delete the old .bib file and re-export the updated collection.

Format Your Citations As You Write and Revise

Dropping in-line citations into your document as you go could not be easier. Start by opening up your exported Bibtex file. As with the other components of this process, it’s also essentially a plain-text file, with no hidden formatting. You’ll see that each of your bibliography items is recorded as a record with several fields, like this:

 @book{gransden_historical_1974,
    location = {Ithaca, N.Y.},
    title = {Historical writing in England},
    isbn = {0-8014-0770-2 978-0-8014-0770-3 0-8014-1264-1 978-0-8014-1264-6},
    number = {Book, Whole},
    publisher = {Cornell University Press},
    author = {Gransden, Antonia},
    date = {1974},
    keywords = {chronicle, laercost}
}

You see that the record starts with an indication of the type of source (book, article, etc.). The next piece of information in each record is the most important for our purposes, the one that reads grandsen_historical_1974. This is called the citekey for this record–the unique ID that allows pandoc to differentiate it from every other record in the file. Dropping a citation into your document is very straightforward:

  1. Open a square bracket: [
  2. Type an @ symbol followed by the desired citekey
  3. Add any needed text or page numbers before and after the citekey.
  4. Close the bracket.

So, a normal citation would look like this:

Here is a brilliant sentence that requires a citation.[@grandsen_historical_1974]

That’s it. That citekey will follow that sentence wherever it goes.

If you want to add other information, such as discursive text, page numbers, etc., just add that text where you need it. Separate the page numbers from the citekey with a comma:

Here is a brilliant sentence that requires a citation.[If you need some discursive text, just add it inside the brackets before the citekey. See @grandsen_historical_1974, pp. 147-52]

If you need to include more than one citation, separate the citekeys with semicolons:

Here is a brilliant sentence that requires a citation.[Some discursive text. See @grandsen_historical_1974, pp. 7-9; also @anzaldua_borderlands:_2007, pp. 153-8]

Easy, yes? Once again, these citation placeholders will follow your text no matter what you do with it, just like any other kind of markdown formatting.

Grab The .csl File For Your Documentation Style

This might seem a little tricky at first, but it isn’t. The .csl file is simply a file that contains all the instructions for how final citations should be formatted according to a particular style (Chicago, APA, etc.). You can get a pre-made .csl file for most documentation styles from the Zotero Style Repository here. Simply find the style you want, click the link, and download the file into the same directory as your main text file and your bibtex file.

And Now, the Magic: Creating Fully-Cited Documents With One Swish:

Okay, now that we’ve got all the materials together, it’s time to tell Pandoc to do its thing. Let’s start with a sample markdown document with some citation placeholders included:

MD_Citations

 

Now we just need to tell Pandoc to put all the pieces together. At this point we should have three files in our document directory: Our text file, the .csl file for our chosen documentation style (I normally need the Chicago Full-Note and Bibliography style, so I’ve downloaded that one for this example), and the bibtex file containing our references, like so:

MyDocument.md
MyBibtexFile.bib
chicago-fullnote-bibliography.csl

Here comes Pandoc. Fire up a terminal and type:

pandoc MyDocument.md -s -S --bibliography MyBibtexFile.bib --filter pandoc-citeproc --csl chicago-fullnote-bibliography.csl -o MyFormattedDocument.odt

Let’s break down what each part of the command is doing:

  • pandoc simply invokes the pandoc app.
  • MyDocument.md tells pandoc which markdown document to process
  • -s -S These two tags tell pandoc to create a standalone document file, and to use smart quotes, respectively
  • --bibliography MyBibtexFile.bib Points pandoc to your bibliography file.
  • --filter pandoc-citeproc tells pandoc to use the pansdoc-citeproc tool to process your citation placeholders
  • --csl chicago-fullnote-bibliography.csl tells pandoc to format your citations according to the information in the chicago fullnote bibliography csl file, and points it to that file.
  • -o MyFormattedDocument.odt tells pandoc to output to an .odt (LibreOffice, OpenOffice) file named “MyFormattedDocument.odt” If you want a different file type, just change the extension (.docx for Word, .pdf for Portable Document Format, etc.)

And here’s the reveal, your fully formatted document:

Screenshot_Chicago

But let’s say (alas!) one publisher turns down that manuscript. You want to submit it elsewhere, but the new journal to which you’re submitting requires MLA rather than Chicago style. No problem–and no need to do anything with my original text document. I simply download the MLA 8th edition .csl file from the Zotero Style Repository, and change the pandoc command to use that stye:

pandoc MyDocument.md -s -S --bibliography MyBibtexFile.bib --filter pandoc-citeproc --csl modern-language-association-8th-edition.csl -o MyFormattedDocument_MLA.odt

And my output looks like:

Screenshot_MLA

There it is! While this tutorial takes a little longer to explain the process than some of the others, I hope it’s apparent how easy and muss-free this can be if you set it up as your workflow from the beginning. As always, I’m glad to field questions about the process. Happy writing!

The Solarized Writer’s Desktop

elegant_solarizedIn two more weeks, I end my post-heart-surgery medical leave and go back to doing what I love the most: being with students, teaching Chaucer and Shakespeare, and studying the literature and culture of medieval Britain.

It’s going to be a bit of a transition after three months of focusing on healing, so I’ve been spending my weekend doing a little preliminary “gearing up”, sort of preparing myself psychologically to head back to work, now that my body can actually handle it again.

Since part of my job involves doing some pretty heavy-duty research and writing, part of that prep has involved thinking about ways to work on those projects, especially the writing, with a high degree of focus. The computer is of course both a blessing and a curse in this regard: it’s a necessary tool, to be sure, but it can also be mightily discracting, not only in the sense of offering online diversions (social media, etc.) but visual clutter as well. Our operating systems and browsers seem to be increasingly geared to be platforms for selling us things, which means notifications and message-boxes of various kinds can really proliferate (Windows 10, for example, drives me insane in this regard). With multiple windows and toolbars, computer screens can become a cacaphony of clashing colors and notification sounds, none of which are particularly conducive to concentration.

So, being a bit of a nerdy sort, and also someone who cares about recycling older gadgets to reduce e-waste, I decided to resurrect an old desktop computer to use as a dedicated writing machine. The idea was to make the interface as distraction-free as possible, visually elegant, and conducive to long work sessions while minimizing eyestrain.

There are of course many “distraction free” writing apps out there that try to fill this kind of niche (such as WriteMonkey and Writeroom), but I’ve never found them quite right for me: as an academic, I rarely write only what springs spontaneously to mind: I’m constantly looking at previous drafts of what I’ve written, paging through electronic notes, PDF files of articles, etc. I generally also need some consistent information, such as the time (so I’m not missing my next class!), my to-do list, and basic weather information. I also wanted to capacity to play and control music right on the desktop, since that usually helps me tune out other ambient noise. Dedicated distraction-free writing tools accomplish their task my simply filling the entire screen and blocking out everything else; I wanted a solution that would not blocking things out, but rather byu displaying only what I want. So, my idea was to set up a system that would be as spare and distraction-free as possible without cutting me off from all the information and resources I generally need to manage while I write.

The result is what you see in the screenshot above. I’ll explain the techy bits of this below for anyone interested in duplicating or improving upon my efforts, but here’s what you’re looking at:

  • To create a very customizable system that runs quickly on an older computer, I used the Linux operating system, specifically the Xubuntu Linux flavor. Xubuntu is designed to be easy-to-use with a minimum of system resources while still being a fully-featured desktop environment. A similar arrangement should be possible with Windows and MacOS, using a somewhat different set of applications and tools.
  • To make the whole thing very eye-friendly, I based all the colors on the solarized color scheme developed by Ethan Schoonover. According to Schoonover:

Solarized is a sixteen color palette (eight monotones, eight accent colors) designed for use with terminal and gui applications. It has several unique properties. I designed this colorscheme with both precise CIELAB lightness relationships and a refined set of hues based on fixed color wheel relationships. It has been tested extensively in real world use on color calibrated displays (as well as uncalibrated/intentionally miscalibrated displays) and in a variety of lighting conditions.

I’ve found this scheme to be truly conducive to long periods of staring at a screen. My idea was to integrate all aspects of the desktop within this scheme, so that both the desktop and its various elements (wallpaper, window borders, application backgrounds, music player, system information display), as well as all the applications I need, share this same set of soothing colors.

  • I set up the system to use two monitors: one directly in front of the keyboard that’s entirely dedicated to the text I’m currently writing (this is the lefthand side of the screenshot above, which combines the images from both monitors). Just to my right, the second monitor is dor displaying system and other information (weather, time, music), and also for displaying and managing ancillary documents like notes, images, and PDF’s. This keeps the focus on the main text while also keeping all those other materials ready to hand.
  • As I’ve mentioned elsewhere on this blog, I love to write using plain-text tools, for a host of reasons that I explain here. My editor of choice for composing is an oldie-but-goody well-known to programmers but less so to writers, known as Vim. Vim is an old-school editor designed to run in a good old-fashioned terminal. There’s a definite learning-curve involved in using Vim, but it’s been worth it for me, as Vim has become about the fastest and most distraction-free tool for bare composition I’ve ever found. Since Vim runs in a terminal window, I simply have the lefthand monitor set up to display that single terminal window, made almost transparent, against the solarized background. This keeps the editor in the visual center of the monitor at all times, keeps my hands on the keyboard where they belong, and encourages a focus on what I’m writing.
  • The most prominent element on the righthand monitor consists of a system monitor tool called conky, set up to display the time, weather information, and basic system information.
  • To the left of the system information display, there’s a music player called ncmpcpp. This is another tool designed to run in an old-fashioned terminal. The beauty of this is that I can position a transparent terminal on the screen, and the music player becomes a background element of the desktop, always available but never in the way. Console applications also take up much less in the way of system resources than GUI apps, which helps to keep this older machine running smoothly.
  • The final element is my todo list, which I manage via a wonderful, elegant application by Gina Trapani called todo.txt. It’s basically a fancy way of keeping you todo list in a plain text file while still being able to manage it nicely, add and delete items, set contexts, etc. This is another console application, so it runs in another small, transparent terminal window right underneath the system information display.
  • Note management is a little more complex than simple composition, since it involves keeping track of many different small files at once (I keep my reading and thinking notes in small, plain-text files, organized into directories), so I use a more robust text editor for note management so I can easily switch between notes and have several note documents open at once. I accomplish this with the excellent Sublime Text. Note that Sublime Text (the other window in the right-hand part of the screenshop above) displays both my note documents in tabbed or tiled format, with a left-hand panel that shows the directory tree of my note directory, making everything readily accessible.

Those are the basics! Feel free to let me know what you think or suggest improvements. I’m also glad to work with the less-techy to set up similar arrangements of their own.

A few more details and resources:

Solarized Wallpaper:

Ncmpcpp documentation:

https://wiki.archlinux.org/index.php/ncmpcpp

Solarized Numix theme for XFCE:

https://github.com/gsnewmark/numix-solarized-dark-gtk-theme

Conky Harmattan theme:

https://github.com/zagortenay333/Harmattan