Fork me on GitHub

Thunderdell and BusySponge

Joseph Reagle

Would you like a bibliographic system that was a simple and powerful complement to the way you take notes? The Thunderdell scripts are intended to make sure the important task of engaging a text is not hindered by bibliographic constraints. Using the power of the Freeplane (1.5+) mindmap application you can quickly tag and outline a source and trivially indicate annotations, paraphrases, and excerpts with simple keyboard shortcuts; you can outline, link to Web resources, and include images and tables—anything you can do with a mindmap and HTML. The underlying data is accessible XML and Thunderdell transforms this into pandoc’s citeproc yaml, biblatex, or Wikipedia’s citation template.

Thunderdell is written in Python and requires some familiarity if only to specify the location of your root mindmap. I’ve used this on Linux and Mac, but I believe it could be made to work on any platform as all dependencies (including lxml, dateutil, and requests) are cross-platform. The scripts are available for download.

The command-line BusySponge micro-blogger and Web scraper is also included and can be used to populate your mindmap with bibliographic data about Web resources.

You can download the scripts from https://github.com/reagle/thunderdell.

(Thunderdell is the name of the “simple and powerful” giant from Jack and the Beanstalk who would declare “Fee-fi-fo-fum.” What I used to call “Freeplane Extract” (fe) is just a collection of scripts, such as “thunderdell.py”. BusySponge is just that, a way to quickly capture, log, and annotate things on the Web.)

Thunderdell Features

Thunderdell’s underlying and export data are both in open formats.

It takes advantage of powerful mindmap functionality:

Bibliographic maps are searchable from the command-line or via the Web. Results of queries include the appropriate citation with year and page number.

Using thunderdell.py

─➤  thunderdell.py -h
usage: thunderdell.py [-h] [-a] [-b] [-c] [-D] [-i FILENAME] [-k] [-F] [-j]
                      [-l] [-o] [-p] [-q QUERY [QUERY ...]] [-s BST] [-T] [-u]
                      [-V] [--version] [-L] [-w] [-y]

Outputs YAML/CSL bibliography. Note: Keys are created by appending the first
letter of first 3 significant words (i.e., no WP:namespace, articles,
conjunctions or short prepositions). If only one word, use first, penultimate,
and last character.

optional arguments:
  -h, --help            show this help message and exit
  -a, --author-create   create author for anon entries using container
  -b, --biblatex        emit biblatex fields
  -c, --chase           chase links between MMs
  -D, --defaults        chase, output YAML/CSL, use default map and output
                        file
  -i FILENAME, --input-file FILENAME
                        mindmap to process
  -k, --keys            show biblatex keys in displayed HTML
  -F, --fields          show biblatex shortcuts, fields, and types used by fe
  -j, --JSON-CSL        emit JSON/CSL for use with pandoc
  -l, --long-url        use long URLs
  -o, --output-to-file  output goes to FILENAME.yaml (BOOLEAN)
  -p, --pretty          pretty print
  -q QUERY [QUERY ...], --query QUERY [QUERY ...]
                        query the mindmaps
  -s BST, --style BST   use biblatex stylesheet (default: apalike)
  -T, --tests           run tests
  -u, --urls_online_only
                        emit URLs for online resources only
  -V, --verbose         Increase verbosity (specify multiple times for more)
  --version             show program's version number and exit
  -L, --log-to-file     log to file thunderdell.py.log
  -w, --WP-citation     emit Wikipedia {{citation}} format which can be cited
                        via {{sfn|Author2004|loc=p. 45}}. See:
                        http://en.wikipedia.org/wiki/Template:Cite
  -y, --YAML-CSL        emit YAML/CSL for use with pandoc [default]

Thunderdell relies upon particular Freeplane styles you can find in the test files, such as <stylenode TEXT="author" COLOR="#338800"/>. I also provide an example freeplane-config/user.properties with some keyboard shortcuts (keystroke in Freeplane terminology) for easily applying the styles.

A citation node looks like “d=2000 p=Basic Books a=New York, NY”.

biblatex output

  @book{Lessig2000col, 
     annotation = {A seminal work that identifies the authority of software code}, 
     author = {Lessig, Lawrence}, 
     shorttitle = {Code}, 
     title = {{Code:} {And} Other Laws of Cyberspace}, , 
     year = {2000}, 
     address = {New York, NY}, 
     publisher = {Basic Books}, 
  }

YAML output:

- id: Lessig2000col
  type: book
  publisher-place: "New York, NY"
  abstract: "A seminal work that identified the authority of software code in social structure"
  author:
  - family: "Lessig"
    given: "Lawrence"
  custom2: "neutrality.mm"
  issued:
    year: 2000
  publisher: "Basic Books"
  title-short: "Code"
  title: "Code: and other laws of cyberspace"

Wikipedia output:

<ref name=Lessig2000col>
{{citation
| publication-place = New York, NY
| first1 = Lawrence
| last1 =  Lessig
| date = 2000
| publisher = Basic Books
| title = Code: and other laws of cyberspace
}}
</ref>

Busysponge Features

BusySponge permits me to easily log and annotate a URL, DOI, or ISBN to loggers (e.g., mindmap, blogs) with meta/bibliographic data.

Using busy.py

─➤  busy.py -h
usage: b [options] [URL] logger [keyword] [text]

blog codex:    b o [pra|soc|tec] TAGS URL|DOI TITLE. BODY
blog goatee:   b g URL|DOI TITLE. BODY
mindmap:       b m TAGS URL|DOI ABSTRACT
nifty:         b n TAGS URL|DOI COMMENT
work plan:     b j TAGS URL|DOI COMMENT
console:       b c TAGS URL|DOI COMMENT
  's. ' begins summary
  '> '  begins excerpt (as does a character)
  ', '  begins paraphrase
  '-- ' begins note
  'key=value' for metadata; e.g.,
    au=John Smith ti=Greatet Book Ever d=2001 cb=Blogger.com et=cb
    Entry types (et=cb) values must be typed as container shortcut.

positional arguments:
  text

optional arguments:
  -h, --help            show this help message and exit
  -T, --tests           run doc tests
  -C, --container-shortcuts
                        show container shortcuts (cb, cw, cf, ...)
  -K, --keyword-shortcuts
                        show keyword shortcuts  (adv, fem, wp, ...)
  -p, --publish         publish to social networks
  -L, --log-to-file     log to file b.log
  -V, --verbose         Increase verbosity (specify multiple times for more)
  --version             show program's version number and exit